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PREVIOUS STUDIES 


Through the first decade of the present 
century there was relatively little questioning 
‘. of the traditional classical requirements for 
us college entrance. In rg11, the Committee of 
Nine on the Articulation of High Schools and 
Colleges (6) recommended the requirements 
for college entrance which have since been 
more or less standard. These requirements 
consisted of nine specified units, three in Eng- 
lish, two in a single foreign language, two in 
mathematics, one in social science including 
history, and one in natural science. Two addi- 
tional units were to consist of advanced work 
in academic subjects to complete the require- 
ment of a second major of three units. Four 
units were left as a margin to meet individual 
needs. 
In 1919, Colebank (5: p. 149) stated the 
conviction which was growing among more 
liberal school men that 


“No convincing evidence has made cer- 
tain that a four year secondary school 
course, consisting largely of quite formal 
studies in foreign language and mathe- 
matics, taught by the methods that have 
long been customary in preparatory and 
‘ther secondary schools, actually lays a 
better foundation for successful college 
work and for subsequent success in life 
than courses of a different character. Many 
oleges insist on a certain number of units 
' Latin or other foreign language. They 
should be expected to present evidence of 

mvincing nature that the enforcing of 
> requirement enables them to secure 
‘udents with best preparation for college 
vork, such results being due to the educa- 
nal value of the subject.” 


\bout this time a new basis for the selec- 
‘on of students appeared in the form of psy- 
nological and other aptitude tests. The 
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attention of investigators was directed toward 
a number of factors possibly predictive of 
college success. Those chiefly studied were 
intelligence as measured by some test or bat- 
tery of tests, high school scholastic average 
or rank in class, and the amount of credit in 
specified high school fields. 


As might be expected, there was not entire 
unanimity in the findings regarding any 
factor. However, there appeared to be less 
evidence in support of the predictive value 
inherent in the amount of high school credit 
in any field than in support of the predictive 
value of either intelligence or high school 
rank. After a study of the scholastic records 
of 450 freshmen in Syracuse University, May 
(14: pp. 436, 437) concluded that 


“By knowing the intelligence of a stu- 
dent and knowing the time he spends in 
study we can predict his honor points with 
but slightly greater error than we could if 
we knew also the units he offered and the 
average grade he made in them . . . Aside 
from statistical treatment there are other 
considerations that lead us to believe that 
the amount of preparatory school or high 
school work is a very poor index of college 
success.” 


Gowen and Gooch (10), in a study of 927 
graduates of the University of Maine, found 
that, compared on either the amount or the 
quality of the work done in each, high school 
subjects were about equal to each other in 
predictive or training value for the freshman 
year of college. Odell (16) made an exten- 
sive study of about 2000 freshmen in a num- 
ber of colleges and found some predictive 
value with respect to college grades in the 
number of credits in specified high school 
subject fields. However, in almost every case 
the general high school average, the I.Q., or 
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both taken together, were superior to specific 
subject patterns in predictive value. 

In 1923 Gebhardt (g) found no significant 
evidence, in a study of students at the Colo- 
rado State Teachers’ College, to indicate that 
any one subject or group of subjects in high 
chool is of greater value than another as 
preparation for college work. In 1924, Mac- 
Phail (13) selected twenty-seven pairs of stu- 
cents of equal rating on psychological tests, 
of which one member had taken four years 
of high school Latin and the other none. The 
average freshman grades favored the Latin 
members of the pairs but the difference was 
not statistically significant. 

Bolenbaugh and Proctor (1) found little 
difference between the academic achievements 
of two groups of Stanford University men, 
one of which had three or more units of voca- 
tional subjects in high school and the other 
no more than one. Brammell (2) made a sim- 
ilar analysis of the records of 163 freshmen 
entering the University of Washington in 
1925 with five or more vocational units and 
of 353 with no more than one non-academic 
credit. The first year average favored the 
academic group slightly but the four year 
averages were identical. 


Sorenson (19), in a study of the success 
of Northwestern University freshmen, con- 
cluded that three or four years of high school 
Latin were definitely predictive of college 
success. French and mathematics appeared 
to differentiate students less efficiently. Kent 
and Schreurs (12) studied the relation of 
mental alertness scores, rank by quarters in 
high school class, amount of high school Eng- 
lish and mathematics, and general first year 
college grades to success in college English 
and mathematics courses for 228 men and 
296 women at Northwestern University. They 
found the amount of high school English to 
have a slight negative value for predicting 
success in college composition. The amount 
of high school mathematics had positive value 
for predicting success in trigonometry but not 
in algebra. 

Brammell (3), in 1932, reporting on the 
findings of the National Survey of Secondary 
Education in regard to the entrance require- 
ments of 517 colleges and universities, listed 
the average prescription at about ten units. 
In the period from 1924 to the time of the 
‘Survey, 209 requirements in the academic 
fields had been decreased or eliminated. On 





the other hand, 165 had been 
Many provisions for regular admission 


oth. 
than by high school transcript were list 
but that remained the sole method in & 
cent of western colleges and the chief met} 
in most sections. 


The study of the significance of 
elements of the high school pattern 
lege success has not been discor 
Douglass (8) summarized studies up to 1922 
and reported on his own study at the Univer- 
sity of Oregon (7) of 387 graduat 
class of 1930. He found that (8: ; { 
“Not only was there little or no relation | 
tween pattern of high school credits and col- 
lege success, but apparently there is little re- 
lation between the mastery of any particular 
subject and college success.” He concluded 
(8: p. 147) that “The author of this stud) 
feels certain that, a few decades hence, our 
present practice of selecting college entrant: 
on the basis of minimum credits in certain 
fields of high school credits will seem a 
curious and inexplicable anomaly.” Five 
years later, however, he joined with Smith 
(18: p. 695) in reporting their study at the 
University of Minnesota to say “. . . if there 
is any subject in high school the pursuance 
of which tends more than any other study t 
better marks in college, that field is Latin.” 


Whelden’s intensive study of five succes 
sive classes at Yale indicated that greater 
quantities of Latin taken in private schools 
or after entering Yale College led to better 
scholarship in subjects in which there was 
identity or great similarity of elements. But 
(20: p. 497) “There was no evidence at all 
of any value of a more extended study oi 
Latin as an intellectual discipline, serving t 
extend the scope of intellectual capacity in 
whatever field it might be applied.” On th: 
other hand, Ross (17) found that among 50 
students in three classes at Allegheny Colleg: 
those having more high school language di 
better college work. 


Nelson (15) discovered that among 200° 
students at Iowa State Teachers’ College for- 
eign language was the only field in which the 
amount of credit earned in high school showed 
a significant relationship to college achieve- 
ment. Clark (4), in a study of 1431 women 
and 1182 men at Northwestern University, 
found Latin to be the only high school subject 
having any predictive value for college 
success. 
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In a study of first year achiévement of 861 
embers of the entering a 1934 at the 
state University of Iowa, Hammer (11) 

und that college success is more dependent 
ipon the qualities of the student than upon 

. high school subject pattern. Accumula- 
‘on of credit in a specific subject field had 
ttle relation to college success in that field; 
nd general success which may be associated 
with foreign language and mathematics was 
more easily explained in other ways than by 
ssuming the inherent value of the subject as 
. determiner of college success. 


SUMMARY OF PREVIOUS STUDIES 


Investigations of the importance of the 

high school pattern of subjects for academic 
hoices and success may be classified as 
follows: 

1. Studies relating credit in specific high 

hool subjects to general college average for 
1 given number of semesters of college work, 
or to success in certain college fields, or to 
both. 

After due allowances have been made for 
ability differences, few of the relationships 
discovered prove significant, although some 
special claims are made for Latin, French, 
and mathematics. Most of the investigators 
finding such relationships are cautious in 
asserting that they are causal. Even in pair- 
ing studies it is questionable if all important 
factors other than those under consideration 
are perfectly equated. 

2. Studies of the general college standing 
of students offering classical and vocational 
patterns. 

Such studies reveal few differences and 
those small, but those which appear to exist 
favor the classical groups. However, the neg- 
ative evidence from a number of extensive and 
carefully conducted studies in which no such 
differences were found cannot be overlooked. 

3. Studies of the relation of high school 
and previous college patterns to success in 
certain professional colleges or divisions. 

Very little appears to have been done in 
this field. There is some evidence that cer- 
tain high school courses contribute to success 
in engineering but that the pattern is unim- 
portant for law. 

4. Studies of the general intellectual or 
academic quality of students who express 
their greatest interest in, or actually enter 
ertain specialized courses. 
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The results of such studies are contradic- 
tory. Contradictions may be due to the un- 
reliability of reported interests, to differences 
in admission requirements for various profes- 
sional curricula in institutions where studies 
were made, or to differences in techniques 
employed in the studies. Present evidence 
does not appear to establish real differences 
among students preparing for professions for 
which university training is required. 

PURPOSE OF THIS STUDY 

The purpose of the present study was to 
discover the characteristics of the pattern and 
quality of educational experiences which 
might identify each of several groups of uni- 
versity students. Those selected for the study 
were those majoring in English, law, medicine, 
and engineering. The investigation naturally 
divided into two parts, indicated by the ques- 
tions: 1. What are the characteristics of stu- 
dents of each of these groups in relation to 
their relative success in the curriculum 
chosen? 2. How do the characteristics of stu- 
dents in one group differ from those of 
students in the others? 


COLLECTION OF DATA 


Data for the study were secured from sev- 
eral types of permanent student records at the 
State University of Iowa. Most of these were 
located in the registrar’s office. The groups 
chosen were selected as representative of cur- 
ricula which differ considerably and which 
enroll a number of students sufficient to pro- 
vide enough cases without covering too great 
a span of years. The greatest number of 
classes represented in any group was thirteen. 

In this study, students were classified as 
successful if they completed the work of their 
chosen curriculum and received the appropri- 
ate degree. Students were classified as unsuc- 
cessful if they dropped the course before 
beginning the third year, with grades averag- 
ing less than C, or 2.0. This definition was 
used because total grade points equivalent to 
a C average are required for graduation from 
most colleges of the University. Unsuccessful 
students were placed, for purposes of analysis, 
in two supplementary groups, engineering and 
liberal arts. 


COMPOSITION OF THE GROUPS 
The total number of students included in 
this investigation was 1623. Of these, 280 
completed the four year course in liberal arts 
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with a major in English; 251 completed the 
course in law; 317, the medical course, and 
323 received degrees in engineering. The re- 
mainder included 176 students who were un- 
successful in engineering and 267 who were 
unsuccessful in the liberal arts course. 


There were no women in either the success- 
ful or the unsuccessful engineering group. 
Women numbered only two per cent of the 
law and medical groups. Twenty-five per cent 
of the unsuccessful liberal arts group and 71 
per cent of the English group were women. 
A relatively larger percentage of women than 
of men persisted in college attendance beyond 
the second year, since only one fourth of the 
unsuccessful liberal arts group consisted of 
women, as compared to slightly more than one 
third of the entire group entering at the same 
time. 


AcE AT HicH SCHOOL GRADUATION 


The mean age of successful men students at 
high school graduation was about 17.7 years 
in each group. In all groups it was found that 
students who ranked high in their high school 
classes were, on the average, younger than 
inferior students at high school graduation. 
The mean age of women at high school grad- 
uation was below that of men in the same 
group. There were no students of either sex 
in the entire study who were graduated from 
high school below the age of fifteen or above 
the age of twenty-nine. 


S1z—E oF HicH ScHooL ATTENDED 


Size of high school attended appeared to 
have little influence upon choice among the 
university curricula included in this study. 
The school size distributions for the failing 
groups differed so little from the others that 
one may also conclude, in general, that the 
size of high school attended has little effect 
upon college success or failure. Sub-groups 
composed of students from large schools 
(over 600 students) showed an advantage 
over sub-groups from smaller schools in qual- 
ifying examination percentile ranks and _ in 
first year grades in college. This was true for 
all major groups except English, in which the 
large-school group ranked above the small- 
school group (under roo students) but below 
the group from schools of medium size. How- 
ever, these differences tended to disappear 
during the course, and law was the only field 
in which the large-school group showed an 


advantage in final grades. It seems reasonable 
to conclude from this, that although sch 
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in general do differ by size in the preparation 
they give their students for writing entrance 
examinations and doing first year college 
work, such advantages or handicaps do not 
persist throughout the college course. 


RANK IN HIGH SCHOOL CLAssEs 


The law, medicine, and engineering groups 
differed little in the percentage of members 
ranking in each third of their high schoo! 
classes. The ratio of students of these groups 
sectioned by rank in high school class was 
approximately 60:30:10, from the highest 
third to the lowest. The English group aver- 
aged a higher rank in high school, with a 
ratio of 78:20:2. The unsuccessful groups 
resembled each other in this respect, with 
ratios approximating 20:50:30. From these 
ratios, it is evident that there is a relationship 
between class rank in high school and chances 
of success in college. 

It may be deduced from the ratios given 
that rank in graduating class does influence 
the decision of students in regard to college 
entrance. Even in the unsuccessful groups of 
college students, fewer than one third of the 
number had ranked in the lowest thirds of 
their respective high school graduating classes. 
It is also evident that class rank by thirds 
offers no positive assurance of either success 
or failure in any of the areas studied, since 
low-third students were found in each suc- 
cessful group and high-third students in both 
unsuccessful groups. The latter circumstance 
may be explained by the fact that the defini- 
tion of unsuccessful students makes no allow- 
ance for students of satisfactory ability who 
are unable to remain in college or to do work 
of adequate quality because of ill health or 
other factors which cannot be determined. 


DELAY IN COLLEGE ENTRANCI 


The smallest percentage of students delay- 
ing college entrance after high school gradu- 
ation was in the law group, and the highes! 
in the engineering groups. There appeared t 
be no particular reason for the difference. It 
is of some interest to note that about on 
fourth of all students who enter the Univer 
sity were out of school a semester or longe! 
after high school graduation. However, litt 
evidence appeared in this study to indica 
that this delay affected college success in a! 
way. 











QUALIFYING EXAMINATIONS 


[he mean qualifying examination percen- 
ranks for the law, medicine, and success- 
ngineering groups were 63, 62, and 64, 
ectively. This is as near equality as could 
xpected. Variances were also equal. The 

rank of the English group was 71, sig- 
intly higher than those of the other 
ups. That of the unsuccessful engineering 
croup was 42; and of the unsuccessful liberal 
rts group, 31. The differences in the means 
f successful and unsuccessful groups indicate 
that the qualifying examination has some 
value as an index of college success or failure. 
However, the fact that every successful group 
ncluded students ranking in the lowest decile 
n the qualifying examination shows that it is 
far from absolute as a predictive instrument. 


RELATIVE SCHOLASTIC ABILITY 


The differences in the ratios of students 
ranking in the highest thirds of their high 
school classes, in mean qualifying examination 
percentile ranks, and in mean one year and 
two year grade point averages, all point to the 
English group as scholastically superior to the 
other successful groups included in this study. 
There is evidence that this superiority is not 
a result of the preponderance of women in the 
English group in the fact that the men of the 
English group averaged higher on each of 
these measures than the women. The data 
which support the conclusion as to the supe- 
riority of the English group point just as 
definitely to the scholastic equality of the 
law, medicine, and engineering groups. 


COLLEGE GRADES IN REQUIRED SUBJECTS 


With few exceptions, the grade means of 
all sub-groups in required courses in the first 
and second years of college varied from the 
grade means of these groups in all subjects 
taken at the same time in the direction of a 
2.0, or C average. This leads one to question 
whether apparent differences in the abilities 
of student groups in required subjects, as con- 
trasted with their abilities in all subjects con- 
sidered together, may not be misleading. 
C grades may be more common in these 
courses, where large classes and routine in- 
struction make reliable measures of individual 
accomplishment difficult to obtain. A pre- 
ponderance of C grades would have the effect 
of undergrading the superior students and 
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overgrading the inferior, which would explain 
the conditions noted. Achievement of out- 
standing grades would represent unusual 
promise in the field and might be extremely 
valuable in the selection of students in appro- 
priate majors, especially English. 


(CORRELATIONS 


Qualifying examination percentile rank 
was correlated with final grade averages for 
English majors more highly than with grades 
of engineers, .55 and .43, respectively. Cor- 
relations with grade averages in law and 
medicine were smaller, .35 and .33. First year 
grades showed correlations of .86 and .80 with 
total English and Engineering grades, but a 
spurious element is involved in the inclusion 
of first year grades in the total from which 
final averages were calculated. Correlations of 
first year grades with final grade averages in 
law and medicine, involving no spurious ele- 
ment, were .49 and .47. Correlations of two 
year pre-professional grades with final grade 
averages were .50 for law and .53 for med- 
icine. 

Size of high school attended showed no 
more than chance correlation with final grade 
averages in any field. Rank in high school 
class, however, showed significant correlations 
with final grade averages in every successful 
group. Values ranged from .40 in English to 
.25 for law, with that for engineering .26, and 
for medicine, .31. The latter was almost as 
high as the corresponding correlation of qual- 
ifying rank with final grade averages. 


HicH ScHooL SuBJECT MATTER PATTERNS 


Mean amounts of high school English 
ranged from 8.01 semesters for the English 
group to 7.66 for the engineering groups. The 
greatest difference was less than half a semes- 
ter. In language credit the English group also 
ranked first, with a mean of 5.12 semesters. 
The law and medical groups earned means of 
4.89 and 4.82 semesters, respectively; the 
successful engineering group, 3.63; the unsuc- 
cessful liberal arts group, 2.96; and the un- 
successful engineering group 2.83 semesters. 
The maximum difference in this case was two 
and one fourth semesters. 

Mean credit in social studies ranged from 
6.25 semesters for the law group to 5.64 for 
the successful engineering group. Mean math- 
ematics credit covered a range of a semester, 
from 5.85 semesters for successful engineering 
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students to 4.82 for English students. The 
engineering group was high and the law group 
low in mean amounts of credit in science, with 
5.16 and 4.28 semesters, respectively. Credit 
in the non-academic field ranged from a mean 
of 5.14 semesters for the unsuccessful liberal 
arts group to 2.96 for the medical group, a 
difference of more than two semesters. The 
average total of high school credits ranged 
from 31.90 semesters for the law group to 
31.08 for the unsuccessful engineering group. 

When the unsuccessful liberal arts group is 
compared with the three successful groups 
which began their courses in the College of 
Liberal Arts, it is seen that the mean subject 
matter pattern of the former group differs 
notably from the others only in the smaller 
amount of credit in language and the greater 
amount of credit in non-academic subjects 
earned in high school by the unsuccessful 
group. These differences were about two 
semesters of credit each. None of the differ- 
ences in mean amounts of credit in any field 
among the three successful groups was as 
great as one semester. If the engineering 
groups are added to the comparison, the great- 
est difference in mean amounts of language 
credit is extended to two and one fourth 
semesters and that in mathematics to more 
than a semester. Credit in non-academic sub- 
jects is still within the two semester range. 

Among all the groups considered, high 
school credit varied greatly only in the lan- 
guage, the mathematics, and the non-academic 
fields. The English, law, and medical groups 
earned the highest mean amounts of credit in 
language. The unsuccessful groups were char- 
acterized by greater amounts of non-academic 
credit. The engineering groups averaged more 
mathematics credit, and the law group a small 
amount more of social studies credit. The 
engineering and medical groups took, on the 
average, more high school science than the 
other groups. 

While these differences indicate that pos- 
sibly some preparation was made in high 
school for the curricula selected in college, 
both individual and sub-group variations in 
each successful group equalled or exceeded 
the differences in the means of the major 
groups. In some cases, the pattern of prepa- 
ration for superior students within a group 
differed from that of the lower ranking stu- 
dents of the same group in the same manner 
as that of the entire group differed from those 
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of other groups. This was not universally 
true, however, for the high ranking students 
of the medical group averaged less credit jp 
high school science than the low rankine 
medical students. While high ranking sty. 
dents in English earned more credit in {or- 
eign language than those ranking low, both 
sub-groups averaged less than the English 
group as a whole. 


From the data presented, it appears that 
there is no pattern of high school subject 
matter that prepares students for success in 
one of the fields covered by this study mor 
definitely than for success in any other. The 
sole possible exception to this is in the case 
of additional mathematics and science for 
engineering students. On the other hand, it 
appears that greater amounts of language and 
mathematics credit, and smaller amounts of 
science and non-academic credit distinguish 
to some extent, the better from the poorer 
students in any field. Again, science for the 
engineering group must be excepted. Amount: 
of credit in high school English and socia! 
studies appear of small significance, possibly 
because there was little variation in credit in 
these fields. 

Relatively large curricular offerings in |an- 
guage and mathematics, coupled with reduced 
college entrance and high school graduatio: 
requirements in these areas, and small require- 
ments, along with increased offerings, in non 
academic subjects, account for the possibility 
of greater variation in the amounts of credil 
in these fields offered for college entrance 
Since greater variation does occur in thos 
fields in which it is possible, it seems reasor 
able that greater weight should be given t 
the small variations noted in other fields. 


On this basis, it appears that students enter- 
ing engineering have, on the whole, made some 
special preparation for this curriculum by tak 
ing additional mathematics and science ‘1 
high school. Students who are unsuccessiu! 
in engineering have, on the whole, less prep:- 
ration in mathematics than those who aie 
successful. Law students, on the other hand. 
have tended to take more language and socia! 
studies than the average for all students 
medical students more language and science 
and English students, more English and lan- 
guage. Even in the successful groups, there |: 
some evidence that the better students show 
greater differences in this respect than the 
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poorer. It should be kept in mind, however, 
that these differences, while consistent, are 
probably too small to have statistical signifi- 
cance, and might be accounted for by chance. 


VARIATIONS IN HIGH SCHOOL SCHOLARSHIP 


An additional matter of interest in connec- 
tion with the high school subject matter pat- 
tern is the number of students ranking higher 
1 lower in each high school field than in all 
high school subjects t?ken together. The 
sreatest number of high ranking students in 
the English major group was in the English 
field in high school; in law, social studies; in 
the suecessful engineering group, mathe- 
matics; and in both unsuccessful groups, non- 
academic. The greatest number of low high 
school grades for the English major group was 
in mathematics, with the greatest net number 
in science. In law, the most low high school 
grades were in mathematics; in medicine, lan- 
guage; in both engineering groups, English; 
and in the unsuccessful liberal arts group, 
mathematics, with the greatest net number in 
language. These results may show a trend to- 
ward better or poorer work in certain high 
school fields for members of different univer- 
sity groups, but in every successful group each 
high school field was represented by both high 


and low grades. 
GENERAL CONCLUSIONS From TuIs STUDY 


The general conclusions drawn from this 
study may be stated as follows: 

1. The existence of small variations in the 
amounts of credit earned in high school fields 
nm which requirements are high, and greater 
variations in fields with more liberal require- 
ments, indicates the possible presence of some 
relationship between the curriculum pursued 
in the university and the high school subject 
matter pattern. 

While the specific relationships between 
amounts of credit in various high school fields 
and the choice of college curriculum are small, 
they are consistent among the four curricula 
included in this study. Since the possibilities 
of variation in the high school pattern are 
restricted, any variation for a considerable 
group of students suggests the possibility of 
the operation of some selective factor. It 
must not be assumed, however, that any rela- 
tionships which appeared were absolute or 
essential, as differences of this size might well 
occur by chance. 


The relative number of students receiv- 
ing higher or lower grades in certain high 
school fields than their average in all, points 
to the possibility of special relationships be- 
tween certain high schools and college fields. 

The piling up of frequencies of higher 
grades in certain high school fields and of 
lower grades in others to show a trend in the 
distribution of special interest or ability for 
a college group appears worthy of notice and 
may justify further study. However, the 
presence of many high and low grades in 
other high school fields necessitates extreme 
caution in advancing any generalizations on 
the subject on the basis of this study. 

3. The general scholastic ability of the stu- 
dent, as measured by rank in thirds of high 
school graduating class, rank on college en- 
trance examinations, and grades in prelimt- 
nary college courses, appears to be more im- 
portant for success in any of the curricula 
included in this study than specific charac- 
teristics of the high school subject matter 
pattern. 

Every group included in the study offered 
satisfactory evidence for this conclusion. It 
must not be implied, however, that this gen- 
eralization could be applied to predict success 
or failure in an individual case. 


SUBSIDIARY CONCLUSIONS 


In addition to the general conclusions listed 
above, several subsidiary conclusions appear 
tenable: 

1. Successful English students, as a group, 
are superior in general scholastic ability 
to groups of successful students in law, 
medicine, and engineering. 

2. There are no real differences in general 
scholastic ability among groups of suc- 
cessful students in law, medicine, and 
engineering. 

3. Unsuccessful students, as groups, are 
inferior in all measures of scholasti: 
ability used in this study to correspond- 
ing groups of successful students. 

4. The overlapping of measures of scho- 
lastic ability for individuals in success- 
ful and unsuccessful groups, and for 
individuals ranking high and low scho- 
lastically within successful groups, indi- 
cates that indices of achievement in any 
of the curricula included in this study 
are extremely unreliable when applied to 
individuals. 
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5. There was some evidence in this study 
that the size of high school attended 
may affect relative rank on qualifying 
examinations or in first year grades. 
However, any advantage resulting from 
graduation from a larger high school is 
reduced considerably or even eliminated 
by the end of the university program. 

6. Rank by thirds in high school graduat- 
ing class, qualifying examination percen- 
tile rank, and grades earned in the first 
and second year of college show, for all 
groups, correlations of an approximate 
range from .25 to .50 with averages of 
all grades in the curriculum. 
Since students ranking in the lowest 
thirds of their high school classes did 
not appear in either successful or unsuc- 
cessful groups in as large proportion as 
one third, it may be concluded that rel- 
atively fewer of these students than of 
students ranking higher in their high 
school classes enter college. 

8. The proportion of women in the unsuc- 
cessful liberal arts group was smaller 
than in the entire entering group, indi- 
cating a relatively lower college mortal- 
ity for women than for men. 

g. The presence, in all successful groups, of 
considerable numbers of students with 
educational characteristics quite similar 
to those of failing groups suggests that 
some quality, which might be called per- 
sistence, not revealed by any of the edu- 
cational measures recorded in this study, 
may be an important factor in college 
success. 


~I 


GENERAL IMPLICATIONS OF THE STUDY 


From this study and others in the area 
which have been reported, it appears reason- 
ably safe to point to a few generalizations 
concerning the relationship of high school and 
college subject matter patterns as probabil- 
ities, even though complete proof may be 
lacking. 

In the first place, students who take more 
high school language and mathematics are 
better students, as a group, at least in the first 
years of college, than those who take less. 
This may be explained in four ways: 1. A 
difference in the subject matter which gives 
inherently better background for any scho- 
lastic task to the first group; 2. Differences 
in the learning requirements and in the aca- 





demic competition encountered which cd; 
better study habits in the first group 
selective factor, operating at the high 
level, to give students of greater natur 
ability the first type of preparation 
4. The weighting of certain measures of abi). 
ity and achievement at the college lev: 
matters not properly related to the immediat 
scholastic task, to favor the first 
preparation. 

Very likely each of these conditions is oper- 
ative on some occasions and to some ¢ 
The writer believes that the operati 
guidance methods at the high school eve! 
tends to place better students in academi 
poorer students in non-academic curri: t 
subjects. Since this is true, scholastic compe- 
tition is keener, on the average, in electiy 
academic classes; teachers demand a better 
quality of work; and students receive mor 
stimulation to their best efforts. It is als 
notable that in the academic subjects mental 


activity of a type more closely resembling 
that required in college is emphasized, whi\ 
in many non-academic subjects manual sk 
is given the greater emphasis. Aside from thi: 
there appears to be no reason to questi: 
Whelden’s (20) conclusion that identical 
very similar elements constitute the only gai: 
for college work in the study of such a subject 
as Latin. 


Brammell’s (2) study, as well as others 
shows that there is an original advantage i: 
college for students with academic prepara 
tion. Four year grade averages, however 
show academic and non-academic groups 0! 
equal mental ability to be practically ident- 
ical in scholarship. The extent to which th 
first year grades in college are truly repre- 
sentative of the ability and achievement 0! 
the two groups may thus be questioned. In 
addition to other factors, there may be an 
actual bias, unrelated to academic merit, i 
favor of the academic preparation, operating 
more strongly in early than in later years 0! 
the college course. Some of the pairing studies 
seem to contradict this, and the statement 
must be qualified according to one’s estimat: 
of the reliability of such studies. 


Another question of importance in this area 


is that of specialized abilities or aptitudes 


These do exist, of course, but it is quite pos 


sible that they are emphasized beyond the 


point justified by their true importance. From 
this study, it appears that the majority ©! 
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in law, medicine, or engineering 
have entered any of the three curricula 
practically the same chances of success 
‘the one they did enter as far as their 
to meet the requirements of the course 
erned. Interest, of course, is another 
‘er. It must also be noted that these state- 
e made in relation to success in pro- 
| schooling, not professional practice, 
h may differ. However, it seems safe to 
se a student with a strong professional 
terest to follow it, provided he is capable of 
wl hieving success in any profession at the 
me le vel. 
One of the most impressive findings of the 
present study, in the opinion of the writer, 
the success, in each group, of students who, 
y every measured index, should have failed. 
One may still question whether they should 
e been permitted membership in their pro- 
fessions, and what their ultimate success may 
be. However, it is evident that these students 


: possessed some combination of personal qual- 
my ties which enabled them to survive academic 
th rebufis which would have discouraged most 
sti students. The writer believes, on this evi- 
, lence, that one should be very cautious in 
gai 


cen stating dogmatically that any given profes- 
sional goal is impossible of achievement for 
any student, although it may be seen as 
hers mprobable. 

The findings of research definitely are not 
such as to make it inadvisable to encourage 
a high school student to follow a major in- 
terest in a non-academic field, even though 


lent 

=a he may later enter college. It does seem ad- 
rn visable, however, that he should be required 
: to exert strong and sustained mental effort in 


| rn the work that he does pursue and that he 
should not be shielded from the results of his 


| . scholastic limitations. A realistic view of his 
rm mental abilities, and training in meeting de- 
¥ feat and persisting in spite of it may well be 
di of greater value to a student of mediocre 
oe ability than any particular pattern of subject 
“ matter he may study. 
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THE STUDY HABITS OF FAILING AND SUCCESSFUL 
STUDENTS IN THE FIRST TWO YEARS OF COLLEGE 


C. W. BRowN 
‘University of California 


INTRODUCTION 


An accurate determination of the contribu- 
tion of correct habits of study to college suc- 
cess must await the development of quantita- 
‘ive methods for measuring the various study 
skills utilized in college. In the absence of a 
better method, the questionnaire technique 
was used in the present investigation in an 
effort to determine roughly the amount of 
variation in study methods to be found 
‘among failing and successful students. The 
results to be presented are merely suggestive 
but they assist in getting a more complete 
description of the factors affecting the stu- 
dent’s ability to do work at the college level. 


CONSTRUCTION AND ADMINISTRATION OF 
THE QUESTIONNAIRE 


\ questionnaire was constructed with items 

vering such topics as the following: 
(1) reading, (2) reviewing, (3) concentrat- 
ing, (4) critical thinking, (5) note taking, 
(6) preparing for examinations, and (7) 
promptness in completing assignments. After 
two revisions the questionnaire contained 64 


items. 


The questionnaire was filled out by 211 
sophomore students on probationary status in 
the College of Letters and Science’ and by 
(20 students in an elementary psychology 
course in the same college. The two groups 
were comparable in the number of semesters 
spent in the University and in the number of 
credit hours of work attempted. Ample time 
was allowed for everyone to answer all items. 


METHOD OF EVALUATING THE RESPONSES TO 
THE QUESTIONNAIRE 


To make certain that there was no overlap 
vetween the two groups of students relative 
to their college scholarship, the grade point 
iverage for each individual was obtained 
‘rom the registrar of the University. Arbi- 
trary limits of —4 grade points? and +-4 

* University of California, Berkeley. 


* Grade points are assigned for each unit of credit as fol- 
lows: A 3, B 2,C 1, DO, E and F —1. 
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grade points were set up, and the question- 
naires of all students whose college average 
fell within these limits were discarded. This 
step, although reducing the number of avail- 
able cases, resulted in two groups with a de- 
cided difference in their average college 
achievement. 

In order to find out if differences existed 
in the methods of study of the two groups the 
percentage in each group giving the same 
answer to each item was computed. The dif- 
ferences between these percentages, and the 
reliability of these differences, were deter- 
mined and used as a basis for evaluating the 
study methods. 

Most of the percentages to be considered 
in the following discussion are for what may 
be considered the psychologically correct an- 
swer. By “psychologically correct answer” is 
merely meant the answer which was given by 
the majority of graduate students in psychol- 
ogy who were asked to fill out the question- 
naire. For many of the items there is no 
“psychologically correct answer”. Disagree- 
ment concerning which of the possible answers 
to an item is correct was expected, but such 
disagreement does not invalidate the findings 
to be reported. 

It is not practicable to include a summary 
of the responses to all of the items in the 
present report. Instead a comparison of the 
failing and successful students will be made 
in terms of the larger aspects of study meth- 
ods already mentioned. Some of the items of 
the questionnaire were related to more than 
one of these aspects, and therefore may be in- 
cluded in more than one of the topics in the 
following discussion. 


READING HABITS 


In the table are found the data necessary 
for the evaluation of the different phases of 
study habits. Columns three and four contain 
the average percentage of each group giving 
the correct answers to all of the items in a 
given phase. In columns five, six and seven 
the critical ratios of the differences for indi- 
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vidual items are distributed according to size 
and separated according to the group favored 
by the difference. 

Twenty items of the questionnaire dealt 
with methods of reading. It can be readily 
seen that there is little difference between the 
answers of the two groups. In a few of the 
items one or the other of the groups gave a 
significantly greater percentage of correct an- 
swers, but on the whole the differences were 
small and favored each group about the same 
number of times. The data do not merit plac- 
ing much confidence in any individual item. 
However, the consistency of the small differ- 
énces in so many items supports the conclu- 
sion that the two groups do not differ signifi- 
cantly in the proportion of correct answers 
made to questions concerning their reading 
habits. In general the percentage of failing 
students reporting use of psychologically cor- 
rect reading habits does not differ significantly 
from the percentage so reporting from among 
successful students. 


HaAsits or CONCENTRATION 


Eight items of the questionnaire were con- 
cerned with different aspects of the student’s 
attempt to concentrate on the material of his 
courses. In seven of the eight items the per- 
centage giving the correct answer is larger for 
the failing groups, but in only one of these 
items is the differ¢nce between the two groups 
statistically significant. The mean difference 
in percentage for the eight items considered 
together is only 6, which is not statistically 
reliable. The findings indicate that correct 
habits of concentration are reported by fail- 
ing and successful students in about the same 
proportions. 


ATTEMPTS OF FAILING AND SUCCESSFUL STU- 
DENTS TO EVALUATE CRITICALLY AND 
ANALYZE THE MATERIAL OF 
THEIR CouRSES 


The percentage in each group giving the 
correct responses to the eleven items dealing 
with critical evaluation of course material is 
given in the table. In six of the items the 
larger percentage was found for the success- 
ful students, indicating a tendency for them 
to use methods of critical thinking more fre- 
quently than the students who are failing in 
their college studies. However, as was found 
for other aspects of study methods, the differ- 
ences tended to be small. Three of the eleven 


differences are statistically significant 
these favoring the failing students. th: 
favoring the successful students. I{ 
items are considered together and th 
percentages computed, the difference 
proportions reporting psychologicall: 
answers is only .2 per cent more for ¢! 
ing than for the successful students. T} 
support the conclusion that the tw: 
are not significantly different in tern 
proportion reporting correct answers 
items on critical evaluation. 


Hapsits or Note TAKInc 


Seven items covered different phas 
note taking in regard to both textbook 
lecture material. The table gives the resy): 
on this phase of study. For the most part th: 
differences were small and statistically insig 
nificant. In one question, the proportion | 
failing students giving the correct answer wa 
significantly greater than the proportion 
successful students. Results from the oth 
items support the contention that there is 
difference in the two groups in respect to t! 
proportion of students reporting the 
correct habits of note taking. 


MeEtTHOps USED IN PREPARING FOR 
EXAMINATIONS 


Results from the seven items dealing with 
preparation for examinations are given in th 
table. Five of the differences favor the fail- 
ing student. This might be expected from the 
fact that repeated failures would tend to stim- 
ulate the failing student to find specific ways 
of improving his performance on examina- 
tions. However, of the four statistically reli- 
able differences, two favor the successful stu- 
dents. Apparently there is no wide discrep- 
ancy in the proportion of the two groups rr- 
porting the use of psychologically correc! 
methods in preparing for examinations. 


METHODS OF REVIEWING 


On five of the ten items dealing with meth- 
ods of reviewing lecture and text materials the 
larger proportions of correct answers were 
given by the failing students and on the re- 
maining five items by the successful students 
Neither group is shown to be using the cor- 
rect methods significantly more frequent) 
than the other in nine of the ten items. De- 
spite the fact that most of the standard crit- 
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4] ratios are larger than 1.5, the results do 
not warrant concluding that either group is 
superior to the other in methods of review. 


PROMPTNESS IN PREPARING ASSIGNMENTS 


Six items were concerned with the question 
of whether the two groups differed in the 
ounctuality with which they completed the 
xssignments of their courses. A larger per- 
-entage of successful students tend to fall 
behind in their assignments, and a larger per- 
centage of the failing students prepare assign- 
ments ahead of schedule, but, in general, 
neither group shows marked superiority in the 
distribution of their study time. 


GENERAL SUMMARY OF THE FINDINGS ON 
THE SEVEN PHASES OF StuDY METHODS 


The object of the present study was to 
determine if the difficulties encountered by 
failing students could be attributed to their 
utilization of unsound study habits. From the 
foregoing comparisons it is safe to conclude 
that, in general, the failing student reports 
using correct study habits as frequently as 
does the successful student. In fact, there is 
slight evidence that more failing students 
than successful students have developed 
sound study habits. No doubt many failing 
students could improve their academic 
achievement by improving their methods of 
study, but from the results of the present 
study it cannot be concluded that the diffi- 
culties encountered by failing students are in 
any large measure attributable to their utiliz- 
ition of unsound methods of study. 


Tur ABSOLUTE PERCENTAGE OF FAILING 
STUDENTS UsINnGc CORRECT 
Stupy Hasits 


In the foregoing discussion the statement 
made that probably many failing students 
uld improve their scholarship by improving 
their study habits. This is based on the fact 
that the percentages reporting the use of cor- 
rect study methods tended to fall between 
25° and 75%, and that sometimes from 25% 
to 50% reported using methods which can be 
sidered psychologically unsound. These 
‘acts can be shown to occur for items on 
hich there is little disagreement as to the 
ogically correct answer. For instance, 

20 only 43% of failing students had 

ed the practice of glancing through a 

or of examining the main points in 


a chapter, before beginning a careful reading 
of it. There is little question but that either 
of these procedures of getting an overview of 
the whole chapter before reading it, is more 
sound psychologically than to start at the be- 
ginning of the chapter and carefully read it 
to the end. In item 16, in which the student 
is asked if he takes issue with the points of 
view presented in the lectures, only 34% of 
the failing students answered affirmatively. 
Again, in item 14, 34% of the failing students 
reported that they usually made their first 
review of their lecture notes just a day or so 
before an examination, rather than a short 
time after each lecture or during weekly re- 
views. In item 47, 37% of the failing students 
reported that they do not ask themselves 
questions about the material of an assignment 
while they are reading it. 

The answers to the items just presented, 
together with similar answers to many other 
items, indicate that many of the students fail- 
ing in their courses are not practicing sound 
study habits. No doubt improvement in their 
methods of study would be reflected in better 
grades. However, the same indictment can be 
made against the study habits of successful 
students, for large percentages of these stu- 
dents also are utilizing faulty study methods. 


LIMITATIONS OF THE RESULTS FROM THE 
QUESTIONNAIRE METHOD 


In interpreting the foregoing results the 
limitations of the method should be kept in 
mind. There was no way of determining the 
accuracy of the individual answers, and there- 
fore the reports may not describe the actual 
practices and habits of the students. It can 
be contended that the failing students would 
tend to falsify their reports to a greater de- 
gree than the successful students because of 
the pressure put upon them by their proba- 
tionary status. The truth or falsity of this 
contention can never be established, and 
therefore it must be assumed that the suc- 
cessful and failing students were equally 
truthful in their answers to the items of the 
questionnaire. 

The questionnaire method used here showed 
the number of failing and successful students 
reporting a given study habit. In most in- 
stances it did not obtain an estimate of the 
frequency with which the habit was used by 
the students, nor did it in any way get an 
estimate of the efficiency of the habit when 
it was used. Such limitations of the method 
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should be kept in mind when the results are 2. In none of the larger phases id 
being evaluated. methods investigated was the percentage ;; THI 
failing students using correct habits ‘ 
CONCLUDING STATEMENTS cantly below the percentage of 
1. In general, the percentages of failing Students. 

tudents reporting the use of correct study 3. Large percentages of both fa 

habits were as large as the percentages found — successful students fail to make u 

among successful students. efficient study techniques. 


DIFFERENCES IN PERCENTAGES OF FAILING AND SUCCESSFUL STUDENTS GIVING 1 


PSYCHOLOGICALLY CORRECT ANSWERS TO QUESTIONS ON SEVEN 
PHASES OF STUDY HABITS 


Average Percentage Distribution of Critica tion, © 
Number _ of Correct Answers Ratios for Single Items \f 
Topic of Items for All Items According to Group Favored : 
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THE NATURE OF SOME OF THE DIFFICULTIES OF STUDENTS 
FAILING THE FIRST TWO YEARS OF COLLEGE 


C. W. BRowN AND P. V. LorcrREN 


University of California 


/uction. Numerous investigations have 
hov hat ability to succeed in college is 
nditioned by many factors, not the least 
mportant of which are high school prepara- 
n, reading ability, and scholastic aptitude. 
Most of these studies have used representa- 
tive groups of freshmen or sophomore college 
students. The present study attempts to iso- 
late some of the characteristics of the rela- 
tionships existing between the three variables 
mentioned and college success in a group 
heavily weighted with students who were 
failing to achieve an acceptable level of 
scholarship in their college work. 


Selection of the Subjects. Failing students 
were selected from among those who were 
receiving special attention from the dean of 
their college because of their low scholarship. 
\!| of the subjects used completed their first 
four semesters of work at the University.’ 
lo these were added students who were com- 
parable to the failing ones in terms of the 
number of semesters of work completed and 
the number of hours of course work essayed. 


Measurement of the Variables. From the 
registrar’s office the students’ high school and 
college records were obtained and evaluated 
relative to the quality of the work performed. 
Letter grades were translated into numerical 
scores by assigning the following numbers to 
each grade: A:3, B:2, C:1, D:o, and E or 
F:—1.2 Average grade point ratings were 
determined for each student for his four years 
of high school and his two years of college 
work. In addition, the high school achieve- 
ment was evaluated separately for the follow- 
ing fields of subject matter: English and lan- 
guage, mathematics and natural science, social 
science, and biological science. 

In studying the reading capacity of failing 
students it was desirable not only to learn 
their average reading ability, but to learn if 
any reading deficiencies existed for particular 
types of subject matter. None of the avail- 


University of California, Berkeley. 


*In evaluating the high school achievement, D, E, and F 
vere given the value of zero 


able reading tests for college students gave 
separate scores for the particular subject mat- 
ters which were to be investigated, so it was 
necessary to construct new tests. The reading 
selections chosen for the different subject mat- 
ters were: (1) for English and language, a 
selection on the exposition of the drama; 
(2) for mathematics and natural science, a 
selection on mechanical force; (3) for social 
science, a selection on Egyptian economic 
history; and (4) for biological science, a 
selection on respiration in lower organisms. 
These selections were taken from standard 
college texts which were in use or had been 
used recently in elementary courses in the 
respective fields of subject matter. The texts 
of the tests varied from 3 to 4 mimeographed 
pages. At the end of the reading selection in 
each test there was a set of from 20 to 24 
multiple choice questions dealing with the 
material of the selection. The score for each 
subject matter test was the number of ques- 
tions answered correctly. The score for the 
total reading test was the sum of the scores 
from the individual tests. 

The time allowed for each subtest was 12 
minutes. A preliminary buffer test was used 
to acquaint the student with the method to 
be followed in the remaining four tests. The 
student was required to read over a selection 
once before he began answering the questions; 
thereafter he could refer to the text again if 
he so desired. All of the details concerning 
the length of the selections, number of ques- 
tions, time of testing, scoring, etc., had been 
worked out and standardized previous to the 
present study. 

Scholastic aptitude was measured by means 
of the Ohio State University Psychological 
Examination, Form 19. It consisted of three 
subtests, an analogies test, an opposites test 
and a reading comprehension test. The sub- 
jects were given all the time they desired to 
complete the test, the average time required 
being about two and one-half hours. 

Reliability of the Measures. The reliability 
of the grades of the individuals participating 
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in the present study was computed from the 
following formula:! 


o- 
Xs 


in which o 4? is the mean, for N individuals, of 


o,°. The term o rv is the sampling variance 
1 

of the mean of the errors of measurement for 
any given individual for the full number of 
semesters he has been in school. The term 
o;,° represents the variance of the mean 


semester averages of N individuals. The reli- 
ability of the college grades was .88, and the 
reliability of the high school grades .75. These 
coefficients indicate reliabilities sufficiently 
high for the purposes of the present study. 

The reliabilities of the separate reading 
tests were computed by the odd-even tech- 
nique, using the Spearman—Brown prophesy 
formula. They varied from .79 to .go. The 
reliability of the total test was .92. The reli- 
abilities of the subtests of the Ohio State 
University Psychological Examination, as re- 
ported by the author of the test are between 
80 and .84. The reliability of the total test 
is .92. 

Division of Subjects into Four Scholarship 
Groups. In order to study the three factors 
of high school achievement, reading ability, 
and scholastic aptitude, at different levels of 
college scholarship, the subjects were divided 
into four groups on the basis of their scholar- 
ship for the four semesters of college work. 
The grade point averages for the four groups, 
with the symbol assigned each group, follow: 
Group Fr consisted of the lowest failures, 
having a grade point average of .44 and merit- 
ing dismissal from the University; Group F2 
included students who did not merit dismissal 
but who were deficient in scholarship with a 
grade point average of .85; Group Cr was an 
average group with a scholarship of 1.18 grade 
points; and Group C2 was a superior group 
with an average grade point standing of 1.98. 
The range of achievement represented in all 
four groups is from an average of —.30 grade 
points to 2.93 grade points. 

In the comparisons to follow, the perform- 
ance of Group Cr, which is the C-average 
group, is used as a base, and the achievement 
of the other three groups is expressed as a 

‘From ‘‘Validation Against a Fallible Criterior by E. F. 


Curetor ournal of Experimental Education, 1 (19 pp. 








‘ 
9. 


deviation from the mean of the Cr group ; 
terms of the sigma units of the (; distri. 
bution. 
Comparison of the Four Groups in Tery 
of the Average College Scholarship. When thy 
college scholarship of Groups F1, F2, and (; 
is expressed as relative achievement in terms 
of the performance of Group Cr, we find the 
following results: the mean of Group F; | 
5-69 sigma units below the mean of Grow; 
C1, the mean of Group F2 is 2.54 Sigma unit: 
below this mean, and the mean of Group (: 
is 6.15 sigma units above the Cr mean. It }: 
apparent that very large and significant dij. 
ferences in college scholarship exist betwee: 
the mean achievements of adjacent groups, 


Comparison of the Four Scholarship Grou; 
in Terms of the Average Performance in ¢] 
Three Experimental Variables. Fig. 1 repre. 
sents graphically the relative achievement o: 
the different groups in the three variables 
using the performance of the Cr group as the 
point of reference. Each of the failing groups 
is seen to have a performance much beloy 
that of the Cr group, but the amount of th: 


deficiency is somewhat different for the three 


variables. In the C2 group the greatest supe. 
riority is shown in high school achievement 
and the least superiority in reading ability 
and scholastic aptitude. 

Comparison of the Performance oj th 
Four Groups in the Different Types of Sub 


ject Matter in High School. Fig. 2 presents « 
comparison of the relative performance in the 


four kinds of subject matter in high school o/ 
the four scholarship groups. The range 
variation in relative performance in the dif- 


ferent subjects is considerably greater in the 


Fr group than in either the F2 or C2 groups 


The C2 group does not show uniform superi- 


ority to Group Cr in the four subject matter: 


The two subject matters best differentiating 


the failing groups from the C1 group are Eng 
lish and language and social science. The sul 
jects of greatest differentiating capacity be 
tween Groups Cr and C2 are found in th 
mathematics and natural science category. 
Comparison of the Performance of the Fou 
Scholarship Groups in the Reading T: 
Utilizing Different Types of Subject Mati 
Fig. 3 gives a graphical representation of t! 
relative performance of the four groups in ‘ 
four subject matter reading tests. The rang 
of variation is greatest in the Fr group a! 
least in the C2 group. No one of the ot! 
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groups agrees with the Cr group in their 
relative achievement in the different tests. 

The natural science test highly differen- 
tiates the C2 group from the Cr group, but 
between the F2 and Cr groups, and between 
the Fr and F2 groups, the natural science test 
has the least differentiating capacity. The 
poorest performance of the Cr group in both 
high school achievement and the reading tests 
occurred in the field of mathematics and nat- 
sral science. This somewhat general tendency 
toward weakness in this field would increase 
the difference between Groups C1 and C2 and 
decrease the differences between C1 and the 
jailing groups, thus contributing to the con- 
ditions just noted. Another possible reason 
may be suggested. In the natural science 
reading test there is present less of the speed 
factor and more of the power factor than in 
any of the other time-limited tests. This con- 
dition might operate in this test to give the 
failing student a higher performance relative 
to the Cr group. 

Comparison of the Performance of the Four 
Groups in the Subtests of Scholastic Aptitude. 
The relative performance of the four groups 
in the aptitude subtests is given in Fig. 4. 
Variation in relative performance is large in 
both failing groups. The two vocabulary 
tests, opposites and analogies, highly differen- 
tiate Groups F1 and F2. These subtests show 
greater differences between these two groups 
than any of the other individual measures in 
the three variables under study. 


The Pattern of Achievement in the Four 
Types of Subject Matter in High School. In 
each of the variables under study the partic- 
ular measurement used involved several fact- 
ors, and it is important to note the variation 
in achievement among these factors in the 
four groups of subjects. The pattern of 
achievement in different types of subject mat- 
ter in high school is given in Fig. 5. In order 
to facilitate comparison a line has been drawn 
connecting the averages of each group; no 
function is intended. It is seen that the rela- 
tive achievement in the different subjects 
varies from one group to another. The pat- 
tern of Group C2 cannot be considered as 
typical, or to be expected from all levels of 
scholarship, because the higher averages are 
being depressed by the inclusion of large 
numbers of A grades, it being impossible for 
the individual to attain better than an A 
grade. No average in Group Cx is so affected; 
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thus, the curve of this group probably repre- 
sents the expected (normal) pattern of 
achievement. 


The question arises as to whether the varia- 
tions in the curve of Group Cr are due to 
chance, or reflect reliable differences in prepa- 
ration in the different subjects. Standard crit- 
ical ratios were computed for differences be- 
tween adjacent points on each of the curves. 
They form the basis for the following state- 
ments. In Group Cr preparation in mathe- 
matics and natural sciences is significantly 
inferior to preparation in all other subjects: 
therefore, the initial rise in the curve of C1 
reflects a real difference in quality of prepara- 
tion. The change in trend between social sci- 
ence and biological science is not significant. 
The characteristics in the pattern of high 
school achievement in the C-average group 
may now be stated as follows: the prepara- 
tion in mathematics and natural science is 
definitely inferior to the preparation in the 
other types of subject matter; the preparation 
in English and language, although superior to 
that in mathematics and natural science, is 
significantly inferior to the preparation in the 
two remaining subjects; and the difference in 
preparation in the social sciences and biolog- 
ical science subjects is not statistically sig- 
nificant. 

The curve for Group F2 closely follows the 
form of the curve for Cr. The first rise is 
statistically significant (C. R. 3.67). The 
difference between English and language and 
social science has a standard critical ratio of 
1.87 which is strong evidence that a real dif- 
ference in preparation in these two subjects 
might exist. The trend between social science 
and biological science varies slightly from 
that found in the curve for C1, but the change 
is not significant (C. R. .61). It would ap- 
pear that in general, the pattern of prepara- 
tion in high school subjects of Group F2 fol- 
lows rather closely the pattern of Group Cr. 


In the curve of Group Fr the general 
trends again approach those of the Cr group. 
The first rise is statistically significant (C. R. 
2.59). The second rise corresponds in general 
direction to the change in the other curves 
but is not significant (C. R. 1.27). The third 
portion of the curve, the rise between social 
science and biological science, although not 
significant (C. R. 1.49), is more pronounced 
than found for Group F2. With certain minor 
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variations the pattern of achievement in 
Group Fr corresponds to that of Group C1. 


The present data are not sufficient for sta- 
tistically determining the degree to which the 
variation in achievement in the different high 
school subjects conditions achievement in col- 
lege; but regardless of what role the pattern 
of achievement plays in college study, it may 
be safely concluded that this role would be 
about the same for both failing and successful 
students. The principal differences between 
the several curves in Fig. 5 is not a difference 
in trends but a difference in levels of accom- 
plishment. The failing groups differ from the 
successful groups in terms of lower levels of 
achievement in all fields of subject matter. 


The Pattern of the Reading Performance 
of the Four Groups. Each of the reading tests 
was made long enough so that only a few 
perfect scores were attained. Also, sufficient 
time was allowed that no individual failed to 
reach and work on the questions. Every 
individual, then, performed within the range 
of the test, and the relative patterns of per- 
formance were not affected by factors similar 
to those operating in the evaluation of the 
high school preparation. Inasmuch as the raw 
scores on the various reading tests are not 
directly comparable, the average performance 
of each group was expressed as a deviation 
from the mean performance of Group Cr, 
using the sigma of C1 as the unit of measure- 
ment. Fig. 6 presents graphically the pattern 
of achievement of the four groups. The pat- 
tern of C1 appears as a straight line at the 
value 0.00. Any divergence from a straight 
line in the curves of Group F1, F2, or C2, can 
be considered an indication of a variation in 
pattern from that of Group Cr. The lines 
connecting the scores of a given group are 
drawn to facilitate comparison, no function 
being represented. 

It is apparent that the pattern of perform- 
ance of the Fr group does not follow closely 
the pattern of Group Cr. The highest relative 
performance made by Group F1 was in the 
natural science test. Using the score in this 
test as a reference point, equality in relative 
achievement in the different reading tests can 
be represented by a straight line drawn par- 
allel to the curve of C1. Any divergence from 
this line will indicate a deficiency in the rela- 
tive reading achievement. It is obvious that 
Group Fr is particularly deficient in the read- 
ing of materials in English and language, and 
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in social science. The tests in these fields were 
the two longest; they utilized vocabularies 
and dealt with concepts that were much sim. 
pler than was true for either the natura! gc. 
ence or the biological science ‘tests. From 
these facts the conclusion readily follows tha; 
the individuals of the Fr group are extremely 
slow readers. The F2 group shows a very 
marked deficiency in reading social science 
material. No doubt this latter deficiency js 
primarily due to the low rates of reading oj 
these individuals. 


SIGMA UNITS OF 
. 
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The Pattern of Achievement in the Schy- 
lastic Aptitude Tests. Since the raw scor 
were not directly comparable, in order to plo: 
the achievement in the different tests on the 
same coordinates, the performances of each 
group were expressed as deviations from the 
means of the Cr group. Fig. 7 presents the 
relative achievement of the four groups on the 
three parts of the aptitude test. As Group F: 
made their highest relative performance « 
the opposites test, the scores on this test wer 
selected as a point of reference. The pattern 
to be expected from successful college students 
can be considered to be the pattern of Group 
Cr. It will be noted that the pattern of C2 
diverges very little from this pattern, but that 
there are differences between the patterns of 
the failing groups and the C-average group. 

From the data at hand it is net possible t» 
determine if these variations in the pattern o! 
scholastic aptitudes, as measured by the Ohio 
State University Psychological Test, contrib- 
ute to differentiating failing and successful 
students in college. However, it is very prob- 
able that such differences of pattern as have 
been noted are too small to be of importance 
in conditioning college behavior. The general 
low level of performance in all of the apti- 
tudes, rather than a difference in the pattern 
of achievement among the several aptitudes, 
seems to be the factor contributing to college 
failure. 
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Rate of Increase in Performance With In- 
crease in College Scholarship. Some notion of 
the variations in the rate of increase in the 
performance in the three variables with in- 
creasing college scholarship can be gained 
from Fig. 1. In all three of the variables the 
most rapid change takes place between the 
levels of the F2 and C1 groups. This finding 
points to the existence of conditioning factors 
which vary in importance at different levels 
of college scholarship. It appears that the 
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power factors, functioning in high school 
achievement, still play an important role at 
the higher levels of college scholarship, as the 
ope of the curve for this variable still has 
a high value between C1 and C2. 

Variations in the Rate of Increase in the 
Different Part Measures of the Three Vari- 
ables With Increase in the Level of College 
Achievement. A re-examination of Fig. 2 will 
reveal an important difference in the rate of 
increase in the high school scholarship at dif- 
ferent levels of college achievement. For all 
types of subject matter the curves change 
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much more rapidly from the grade point level 
.85 to the level 1.18, than at any other levels. 
This change in rate is another indication of 
the depression of the average scores of Groups 
F1 and F2 in each of the four subject matters. 

The true nature of the rate of change be- 
tween Groups Cr and C2 for biological sci- 
ence, social science, and English and language 
cannot be determined because of the large 
number of perfect grades made in these sub- 
jects, especially by Group C2. This factor is 
not found in mathematics and natural science. 
The rate of change in achievement in this sub- 
ject matter probably closely approximates the 
rate of change to be expected in the higher 
levels of college achievement. 

The more rapid change in performance be- 
tween F2 and C1 may be noted for each of 
the four time-limited reading tests by re- 
examining Fig. 3. This graph furnishes fur- 
ther proof that the more rapid change between 
F2 and Cr, noted in preceding analyses, can- 
not be attributed to some subject matter 
factor. Whatever the cause of this difference, 
it seems to be operating in all of the types of 
subject matter analyzed in the present study. 

From an examination of Fig. 4 it will be 
apparent that the change in scholastic apti- 
tude is not constant with equal units of in- 
crease in college scholarship, and that the 
rate of increase varies with the different tests. 
Although the difference in college scholarship 
between Groups C1 and C2 is greater than 
between any other two adjacent groups, the 
increase in the associated scholastic aptitude 
is less than that found between other groups. 
A large number of perfect scores in Group 
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FIG. 4, COMPARISON OF THE FOUR SCHOLARSHIP 


GROUPS IN THE SUBTESTS OF SCHOLASTIC APTITUDE 
USING MEAN OF GROUP Cl AS POINT OF REFERENCE. 


C2 would account for this reduction of the 
slope, but since no large number of perfect 
scores on the tests was attained by any group, 
this explanation is not acceptable. A more 
plausible explanation is that the students of 
the C2 group have approached closer to the 
maximal level of scholastic ability for their 
age. Increments of increase in test perform- 
ance will get smaller the closer an individual 
approaches his maximal level. The approach 
to the limits of the student’s scholastic abil- 
ity, as measured by these tests, not the 
shortness of the tests, probably accounts for 
the reduction in the slope of the curve at the 
upper levels of achievement. 

From the foregoing findings it is obvious 
that equal increments of increase in college 
scholarship are not associated with equal in- 
crements in performance on most of the part 
measures of the three variables being studied. 
Increase in college scholarship from a failing 
level to an average level is associated with 
the greatest increase in performance in most 
of the measures utilized in the present 
investigation. 
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Concluding Statements. From the preced- 
ing analysis of the difficulties of students 
failing the first two years of college, the fol- 
lowing conclusions may be drawn: 


1. In the three variables of high school 
achievement, reading ability, and scholastic 
aptitude, the most inferior performance of 
failing students is in the variable of high 
school achievement. 

2. Failing students achieve their highest 
relative performance in the variable of read- 
ing ability. 

3. When high school achievement in the 
four fields of subject matter is examined, the 
failing students are found to give their lowest 
relative performance in English and langu- 
age, and social science; and their highest 
relative performance in mathematics and 
natural science, and biological science. 

4. In the reading tests utilizing different 
types of subject matter the failing groups 
achieved their lowest relative performance in 
reading English and language material, and 
their highest relative performance in reading 
material in natural science. 

5. In the scholastic aptitude tests the 
lowest relative performance of the two fail- 
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ing groups was made in reading compre- 
hension. 

6. The range of variation in performance 
in the different part measures of each of the 
three principal variables tended to be greater 
in the failing groups than in either of the 
successful groups. 

7. The pattern of achievement of the fail- 
ing groups in the different high school sub- 
iect matters did not vary greatly from the 
pattern of the successful groups. 

8. In the reading tests and scholastic apti- 
tude tests there was some evidence that the 
pattern of achievement of the failing groups 
differed from the pattern of the successful 


groups. 
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9g. The most outstanding difference in the 
performance of the failing and successful 
groups was not a difference in pattern of 
achievement, but rather a difference in level 
of achievement, the performance of the fail- 
ing groups being depressed in all of the 
measures studied. 


10. The rate of increase in performance 
associated with increase in college scholar- 
ship was found to be higher for all measures 
between the levels of .85 and 1.18 grade 
points than between any other levels investi- 
gated. This most rapid change falls between 
the level of scholastic failure and the level 
of scholastic success. 











A STUDY OF THE CAVD INTELLIGENCE SCALE AND ITs 
RESULTS ON THE DOCTOR’S LEVEL AT COLORADO 
STATE COLLEGE OF EDUCATION!’ 


LORAINE BRUCE 


Pampa Senior High School 
Pampa, Texas 


1. PROBLEM 


The object of this study is to determine 
the homogeneity of the I. E. R. Intelligence 
Scale CAVD, Levels M, N, O, P, and Q, and 
its validity for the purpose of predicting 
achievement at the doctor’s level in the Colo- 
rado State College of Education and to make 
comparisons of the abilities of students on 
the master’s and dwctor’s levels in order to 
make a more adequate selection of candi- 
dates for work on the doctor’s level. The 
CAVD Scale is made up of four series of 
tasks: Completions, arithmetical problems, 
vocabulary, and directions. The upper five 
levels M, N, O, P, and Q are for use on the 
upper college and graduate levels. 


2. SUBJECTS AND TESTS 


The students matriculating on the doctor’s 
level from June, 1932 to August, 1938, were 
given the I. E. R. Intelligence Scale CAVD, 
Levels M, N, O, P, and Q, Form 4. In order 
to complete matriculation in 1936 and 1937, 
students on the doctor’s level were required 
to take the American Council on Education 
Cooperative English Test and a battery of 
twelve education tests. English Usage and 
Spelling were the parts of the English Test 
which were used. The battery of education 
tests consists of tests in the following sub- 
jects: Techniques of curriculum making, ele- 
mentary education, history of education, 
mental tests, personnel and guidance, phil- 
osophy of education, psychology of learning, 
research, school administration, secondary 
education, educational tests, and statistical 
methods. The tests were made up by various 
faculty members of Colorado State College 
of Education and were revised by the Per- 


‘Based on Field Study No. 2, Colorado State College of 
Education, Greeley, Colorado, 1940. This study is a continu- 
ation of the study, ‘“‘The Relationship between Scores on the 
CAVD Intelligence Scale and Success in Graduate Work at 
Colorado State College of Education,’’ which was published 
in the March, 1940, issue of the Journal of Experimental 
Education. 


sonnel Department of the college. There are 
two equivalent forms of each test. All the 
items of each test are multiple-choice of foyr 
choices. The number of items in the varioys 
tests range from forty for research to fifty- 
seven for personnel and guidance. Data on 
each test were compiled from the group of 
ninety-one graduate students who first took 
the tests in the summer of 1936. 


3. RESULTS 


Of the students on the doctor’s level wh 
took the CAVD Intelligence Scale, 60 were 
in the graduate school at least two quarters 
and took a minimum of eight quarter-hours 
each quarter. The number of cases through- 
out the study is relatively small but no more 
were available; more reliable results might 
have been obtained from a larger number. 


The reliability of teachers’ marks on the 
basis of the graduate work done by this group 
of 60 students in one quarter is .433 + .105 
(SE). This is a relatively low reliability. I: 
order to compute the reliability of teachers 
marks and other correlations in which teach- 
ers’ marks are involved, the point-hour ratio 
was found. The point-hour ratio was com- 
puted by dividing the total number of points 
by the total number of hours. The points 
were found by multiplying the number oi 
hours of each course by the point values of 
the letter grades. The point value of “A” is 
‘ne bas ee, 23 a “oe”, 3; 
ar’, & 

The product-moment coefficient of correla- 
tion between scaled scores on the CAVD 
Intelligence Scale and the point-hour ratio 
based on the two quarters of the marks from 
which the reliability was computed is .52! 
+ .094. The coefficient of determination is 
.27; this coefficient gives the percentage ot 
variance of teachers’ marks which is associ- 
ated with the CAVD Scale; the percentage 
of variance which is independent of the 
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CAVD Scale is .73. A half correction for 
attenuation was made in which the unreli- 
ability of teachers’ marks was corrected, but 
in which no correction was made for the un- 
reliability of the CAVD Test scores. This 
computation yielded a correlation of .670 
between the CAVD Test scores and the theo- 
retical true marks, which, rather than .521, 
is the measure of the true validity of the test 
when compared with a true criterion. The 
difference of the means of the point-hour 
ratio of the groups making above the 75th 
percentile (N == 49) and below the 25th per- 
centile (N = 15) on the CAVD is .704. 
Since the standard error of the difference is 
138, there is virtual certainty that there is 
a significant difference. 

In addition to a comparison of the results 
of scaled scores on the CAVD Intelligence 
Scale and teachers’ marks, a study was made 
of the relationship of each division of the 
CAVD Scale to teachers’ marks. The 
product-moment coefficients of correlation 
(N == 46) obtained between the total point- 
hour ratio of teachers’ marks and raw scores 
on the various divisions of the CAVD Scale 
are: (a) Completions, .4g1 + .112; (b) 
arithmetic, .136 + .145; (c) vocabulary, 
297 + .134; and (d) directions, .540 + 
105. These correlations show substantial re- 
lationships between completions and direc- 
tions and teachers’ marks, slight relationship 
between vocabulary and teachers’ marks, 
and only negligible relationship between 
arithmetic and teachers’ marks. 


In order to complete matriculation in 1936 
and 1937, students on the doctor’s level were 
required to take the American Council on 
Education Cooperative English Test. The 
parts given were English Usage and Spelling. 
The product-moment coefficient (N — 54) 
between scaled scores on the CAVD Scale 
and scaled scores on the English Usage Test 
is .487 + .104; that between scaled scores 
on the CAVD and spelling, an r of .610 + 
086. These r’s show substantial relationships. 


In 1936 and 1937, students on the doctor’s 
level took Form A of the battery of twelve 
education tests. A product-moment coefficient 
of correlation (N = 48) of .691 + .076 was 
obtained between scaled scores on the CAVD 
Scale and the average of weighted scores on 
the education tests. This correlation indi- 
cates an improvement of 49 per cent over 
the assignment of scores by mere chance and 
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that the forecasting efficiency is 28 per cent. 
When this correlation was compared with 
the one for the students on the master’s level, 
it was found that 97 times in 100, the cor- 
relation for students on the doctor’s level will 
be the higher. 

A comparison of abilities of students on 
the master’s and doctor’s levels was made by 
computing the differences between the means 
and standard deviations and standard error 
of each difference for the following measures: 
(a) Scaled scores on the CAVD Scale, (b) 
first quarter point-hour ratio, (c) second 
quarter point-hour ratio, (d) point-hour 
ratio based on the first two quarters, (e) 
scaled scores on the English Usage Test, 
(f) scaled scores on the spelling test, and 
(g) average of the total weighted scores on 
the battery of education tests. 


The mean of the scaled scores on the 
CAVD Scale made by students on the doc- 
tor’s level is 7.338 + .884 higher than the 
mean made by students on the master’s level; 
the difference of the standard deviations is 
.604 + .625, with the SD of the masters the 
higher. Since the significance ratio (D/ep) 
for the difference of the means is 8.30 there 
is positive assurance that, on the basis of the 
CAVD Scale, the doctors are always supe- 
rior to the masters. When the means of the 
three groups of point-hour ratios were sub- 
tracted, the means obtained from ratios made 
by the doctors were the higher in each case. 
The difference between the means of the 
point-hour ratios for the first quarter is .133 
+ .058. The difference between the means 
of the point-hour ratios for the second quar- 
ter is .265 + .072. And the difference be- 
tween the means of the point-hour ratios 
based on the first two quarters is .221 + 
.o61. The significance ratio of 2.30 indicates 
that, on the basis of teachers’ marks for the 
first quarter, doctors’ students are not always 
superior to the masters. The significance 
ratios of 3.68 and 3.63, for marks for the 
second quarter and for a composite ratio of 
the first two quarters, respectively, indicate 
virtual certainty that, on the basis of these 
marks, the doctors are always superior to the 
masters. None of the differences between the 
standard deviations of marks are significant. 

On the English Usage Test, the means and 
standard deviations of the masters are the 
higher. The difference between the means is 
1.272 + 1.319; and the difference between 
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the standard deviations is 1.056 + .934. 
Neither of these differences is significant. 
The ratio (D/op) of the differences between 
the means and the standard deviations on 
the spelling test indicate even less assurance. 
When the means of the scores on the educa- 
tion tests were subtracted, that of the doctors 
was higher by 4.105 + .755. The significance 
ratio, 5.44, of this difference indicates with 
virtual certainty that, on the basis of the 
education tests, the doctors are superior to 
the masters. The standard deviation of scores 
on the education tests is higher by .643 + 
.534, which is not a significant difference. 

In order to study the homogeneity of the 
upper five levels of the CAVD Scale, all pos- 
sible intercorrelations of its four parts were 
computed for 93 students on the doctor’s 
level. From the correlations, ro, = .378, 
T ca.v = .331, Toap = .251, and fcavo = 
.242, it is evident that the correlation between 
C and A is affected more appreciably by D 
than by V. The net correlation between C 
and V is affected very little when A is elim- 
inated, but D is evidently a factor in the 
correlation. This is shown by the following 
r’s: Toy = .534, lcv.a = -507, I cv.p = .414, 
and r cy.an == .409. The correlation between 
C and D is the highest of all, and indicates 
a substantial relationship between completions 
and directions. From the correlations, rep = 
-553, Top.a == .491, Tcopv = .442, and 
Top.av == .386, evidently both A and V affect 
the correlation between C and D. The cor- 
relation of .193 indicates only a negligible 
relationship between arithmetic and vocabu- 
lary. The partials, ray.c —= —.OI1, rav.p = 
.075, and rav.cp = —.033, denote the fur- 
ther reduction of the correlation between A 
and V when C, D, and both C and D are 
eliminated. The correlation of .327 between 
A and D denotes low relationship. From the 
partials, ranc == -153, Tap.v = .278, and 
Tap.cvy == .156, C is a factor on which both A 
and D are dependent. The ryp = .392 is not 
appreciably lowered when A is partialled out 
(Tvp.a == .355) but the elimination of C, 
which reduces the r to .137, indicates that 
vocabulary, directions, and completions are 
interdependent variables. 


4. CONCLUSIONS 
The I. E. R. Intelligence Scale CAVD, 
Levels M, N, O, P, and Q is a discriminating 
instrument for determining the range of abil- 
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ities of graduate students on the doctor's 
level. 


The correlation between the CAVD Scaje 
and teachers’ marks is relatively low: how. 
ever, this correlation is consistent with the 
low reliability of teachers’ marks on the doc. 
tor’s level. The CAVD Scale is sufficient}, 
discriminating to make it virtually certaiy 
that those students whose scores are above 
the 75th percentile on the CAVD will mak. 
higher marks than those students who 
scores on the CAVD lie below the 25th per 
centile. 


Although they evidently measure differen; 
abilities, substantial relationships were found 
between the CAVD Scale and the American 
Council on Education Cooperative English 
Usage Test and between the CAVD Scale 
and the American Council on Education Co- 
operative Spelling Test. There is a marked 
relationship between the CAVD Scale and the 
battery of education tests. 


On the basis of comparisons of scores on 
the CAVD Scale, teachers’ marks for the sec- 
ond quarter of graduate work, teachers’ marks 
for the first two quarters of graduate work, 
and scores on the education tests, students on 
the doctor’s level are always superior to stu- 
dents on the master’s level. The significance 
ratios of 8.30, 5.44, 3.68, and 3.63 for the 
differences between the means of the scores 
made by students on the doctor’s and mas- 
ter’s levels on the CAVD Scale, the battery 
of education tests, marks for the second 
quarter, and composite marks for the first 
and second quarters, indicate the varying 
degrees of assurance that the doctors are 
superior to the masters’ students. On the 
basis of the results obtained from these 
measures, a scale of values might be set up 
to be used in selecting candidates for the 
doctor’s degree. 


From the results of the intercorrelations of 
the four divisions of the CAVD Scale, the 
language tests (C, V, and D) are measures 
of different traits from those measured by the 
arithmetic test. Such correlation as appears 
between the language tests and arithmetic 
must be due to their common dependence 
upon reading comprehension. Evidently, A is 
measuring something very different from C 
and V and largely different from D. Since 
the divisions of the CAVD Scale do not over- 
lap very much, each part makes a unique 
contribution to the whole. 
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ANALYSIS OF THE EFFECT UPON THE RELIABILITY 
COEFFICIENT OF CHANGES IN VARIABLES INVOLVED 
IN THE ESTIMATION OF TEST RELIABILITY 


TEOBALDO CASANOVA 
University of Puerto Rico 


|. THE VARIANCE OF THE HALVES IN THE 
SpLit-HALF METHOD 


For computational purposes the reliability 
of a test may be defined as the correlation 
between two equivalent forms. For this pur- 
pose two forms are said to be equivalent 
when their means and variances are equal— 
a situation that rarely, if ever, occurs in 
actual practice. As neither the differences 
nor the significance of the differences between 
the statistics of one form and those of the 
other form are usually reported along with 
the reliability of the test, we have been used 
to think about this reliability without much 
concern about the equivalence of the forms 
correlated. When two forms are available 
their correlation is obtained through the use 
of their actual statistics. Otherwise the reli- 
ability is estimated by the split-half method, 
which owes its popularity to the fact that 
only one form is necessary for the purpose. 

In this last method the correlation between 
the two halves of the available form is some- 
times computed from actually observed data, 
and from this the correlation of the available 
form with a hypothetical form is estimated 
through the Spearman—Brown formula. Here, 
even when the standard deviations of the two 
halves of the available form differ, the four 
standard deviations—two of the available 
half and two of the hypothetical half—are 
implicitly assumed to be equal when the 
Spearman—Brown formula is applied. This 
contradiction may be avoided through the 
use of one of the formulas given below. By 
another method in which the reliability is 
estimated in terms of the variances of the 
sums and differences of the available halves, 
the four standard deviations are also assumed 
to be equal. It is obvious, therefore, that the 
use of different methods in the presence of 
actual disparity in the variance of the halves 
leads to different results which make difficult 
the comparison and transmutation of reliabil- 
ities computed under varying circumstances. 


Let the subscripts z and 2 designate the 
scores on the two available halves of the test, 
and the subscripts J and J/ the scores on the 
corresponding halves of the hypothetical 
form. It is necessary to assume that o, == oa; 
and that o, = on. But suppose that o, + o, 
and that o; + oy. It is also necessary to 
assume that the four intercorrelations are 
equal. Let each one of these be known as 7, 
and let X, + X, = S,, and X; + Xn = 
S;. Then, 








. . — (o,? + 20,0, + o2*) 
(Ki + X2)(X 14 Xu) o,? wr 270,02 + os? 
or, 
r(o, + 92)* (1) 

Tsisy — Gs," 

In practice r is the correlation between the 
two available halves, or the reliability of one- 
half of the test, and rs,s; is the estimated 
correlation of the available form with the 
hypothetical form, or the reliability of the 
whole test. 


If 


2r 
(2) 
+r 





Ts\s1 = 


which is the Spearman—Brown formula as 
used in the split-half method. Now S is the 
score on the sum of the two halves, or the 
score on the whole test. Let D be the score 
on the difference between the two halves. 
Then S17 + 2 and D=1—2 2, and 


og’ = o,* + 270,0, + o,”. (3) 
op’ = 0,” — 2F0,0, + 9,°. 


Subtracting, 
eet ae ie 
49,9, 


Substituting this value in (1), 


» gs’ — op (9, + ,)* 
$81 os? 40,0, 
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Since 
(o, + o,)* = (oe, — o,)* + 40,02, 


Tsisi= [i =] p+ — ey (5) 
* o% 


The form of this id be changed to 


T3191> [-2][-aS é (5a) 


(0, + o,)? 











Now, substituting (4) into (2), 


2 (as" a op’) 
sisi ~ 


2 2 ’ 
0s — op + 40,0, 





and as from (3), 
as" + op? = 2 (0,7 + o,*) = 
2 (0, —o,)? + 49%, 


Os 4 — gp’ 
Os =m (oe, —~o.) 





I 
? — gp?) .—_—__— 


2 


== (We —<—<$_——_— 
as — (0, — 9) 


or, 





os? 


rp 
Tais1 |: — a a= (o, Cen . (6a) 


os* 

Formula (1), although never used, yields 
the most nearly correct estimate of the reli- 
ability of the test because it involves the least 
number of assumptions. It is easily seen that 
unless o, ==, its value is smaller than that 
given by (2), if its form is changed to 


r (o, + 9,)? . (1a) 
(9, — 02)? + 20,0, ( +r) 


A common procedure is to find the correla- 
tion between the halves using the observed 
variances and then apply the Spearman— 
Brown formula. The value thus obtained is 
represented by formulas (6) and (6a), as 
may be inferred from their derivation. As the 
four variances are assumed to be equal in the 
derivation of the Spearman—Brown formula, 
there is a contradiction which may be avoided 
by using formulas (5) and (5a) which only 
involve the necessary assumptions that o, = 
o; and that o, = oy. (5) and (sa) are dif- 
ferent forms of (1). 





Tsisi= : 
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Since 
(9, + 02)? > a? 
unless r = 1.00, the value given by (6a) ; 
larger than that given by (5a) unless o, — 0, 
when the two values are equal. 

If o, = o,, each one of these formuls 
becomes 





This formula, derived by John Flanagan: 
is now frequently used, especially when the 
tests are scored in the scoring machine, ly. 
cause the S’s and the D’s are read directly ;; 
the dial. Its derivation assumes that the fou 
ae are equal and its use, in case th 
o, + o, yields a smaller reliability than tha 
obtained from either (5) or (6). But it is no 
necessary to find op if the variance of one. 
half of the test is available. By using (; 
(7) may be turned into 

4o,* di ’ (8) 





Tsist — 

os 
But this requires a knowledge of r. From (2 
i+r I 





2r 'sis1 


Then, 


2 


— 





I 
— ome F . 
r T31s1 


Hence, 


i. in . (9 


2—Tsis1 


Substituting this value in (8), 


20," (10) 
I —_— ~/ . \ 
o8 


in which o,? is the variance of one-half of the 
test, and os? is the variance of the whole test. 
The formula is quite simple and the neces 
sary data are easily available. The scores on 
one-half of the test are also read directly in 
the test scoring machine. It assumes that the 
four variances are equal, and its value & 
smaller than that given by either (5) or (6) 


* Reported in Rulon, P. ? A simplified procedure for ¢ 
termining the reliability of a test by split-halves. Harve 
Educ. Rev., I (1939), p. 101. 
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I]. THE CORRECTION FOR GUESSING 


The Reliability of the Rights and the 
Reliability of the Wrongs. The well known 
formula for correcting the scores in a mul- 
tiple choice test for guessing is, 

, W 
X=R— P 


Ga J 


(11) 





2 
Cr” — 27rrw Cr 
TxiX2 __ 


COEFFICIENT 


effect of changes in the several variables upon 
the coefficient of reliability of scores cor- 
rected for guessing, rx,x».- 


Let 


TWwiWe 2 


=a’. 
TRiR2 


Dividing both sides of (16) by 7x, x2, 


ow" 





a + 


2% 
n—I (n—1) 





TriRe SH» 
oR Trw OR 





in which X is the score corrected for guessing, 
R is the number of right answers, W is the 
number of wrong answers, and n is the num- 
ber of choices. Dichotomous scoring is as- 
sumed throughout. Summing and dividing by 
\, the number of cases, 


My = Ma— Mw ’ (12) 
n—I 
where the M’s are averages. If N =a, 
becomes, 
W ow 
—(a—*) (13) 


Here X ois the true score after correction 
for guessing, Ro is the true score on the 
rights, and Wa is the true score on the 
wrongs. Squaring, summing, and dividing by 
the infinite population in (13), 


X R w 


2 a 
Tx” 'x,X2 == OR TRyR2 — 2?Rw CRVTRiRe 
ow VY 


a 
ow Twiwe 
ua-— I 


(a— 3)" 

Terms of the type ox” rx,x2 represent the 
variance of true scores, and terms of the type 
rx.x» are reliabilities. Squaring, summing, 
and dividing by the population in (11), 


2 


(14) 


TwiWe 


Ow ‘9 ow 
' ( n as 





ox" on” — 27rw ox 


ae 15 
I)- ( ) 
whence ’ 


ow V 


I 
TX X< 


(17) 


ow" 





(a—s)  (8—57 

Now, ‘rw is the correlation between the 
number of right answers and the number of 
wrong answers and it may be expected nor- 
mally to be negative. As dichotomous scoring 
has been assumed throughout, if each one of 
the subjects taking the test attempts to 
answer all the items, raw == —1.00. There- 
fore, assuming that rew is negative, it is 
obvious that 


Tx,:X2 > ,==, OF < friro, 


as 


e> ,=,OrF < Ff. 


In such a situation it may be concluded that 
if the reliability of the rights and that of the 
wrongs are equal, the correction for guessing 
is unnecessary. If the reliability of the rights 
is greater than that of the wrongs, the reli- 
ability of the corrected scores is smaller than 
that of the rights, and the contrary is true 
in the opposite case. 

2. The Variance of the Rights and the 
Variance of the Wrongs. The reliability of 
the wrongs is not usually available, but the 
variance of the wrongs may be easily ob- 
tained. If the standard errors of measurement 





2 
ow tw 1 We 





Or’ TRiR2— 2? rw oRV Trike —TwiWe “+ 


(n—1)? 


~ ial (16) 





or — 27rw OR ~ 


—r° (n—r1)* 


_ This formula will not be generally useful 
in estimating the reliability of scores corrected 
lor guessing because rrw and rw,w. are not 
usually available, but it serves to show the 





of the rights and of the wrongs are assumed 
to be equal, the reliability of the wrongs can 
be eliminated from equation (17). Doing so, 
and letting 
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ow 
OR a P» 
I — Tair 
= 12 
Pn ee (18) 
I —Twiwe 
Then, 
I —frire 
I—_ TwiWe — ry 
p 
or, 
I—_ TRiRe 
TWiWe = j— 2 
p 
and, 
I 
j <n 
'wiw I 
a? == —* 7 — — + ——__. (19) 
TRiR2 p TRike2 


Substituting this in (17) and dividing both 
members of the fraction on the right of (17) 
by oR’, 


I — 2%raw 








I p—1\% I 
I ——, [1 
TX:X0 _ —+( + TRiRe )+—( 
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depends respectively on whether the variance 
of the wrongs is greater, equal to or smalle; 
than the variance of the rights. Thus the role 
of the variance of the rights is similar to that 
of the reliability of the rights, and the role 
of the variance of the wrongs is similar t, 
that of the reliability of the wrongs, both o; 
which would have been fair inferences 4, 
draw from the assumption expressed in equa. 
tion (18). Since the presence of guessing jn 
a test is indicated by the presence of the 
wrongs, at least to the extent to which the 
variance of the wrongs is a measure of the 
quantity and intensity (whether mild or wild) 
of guessing, the above conclusions are jus. 
tified. One of these implies that if the vari- 
ance of the wrongs is exaggerated in compar. 
ison with that of the rights, the reliability of 
the uncorrected scores (that of the rights) 
suffers a shrinkage in comparison with that 


2 
p—! 
) 
TR Re 











TRiR2 p 


7 . (20) 





I—a2r 
RW reams + 


Here it is evident that if ra,x. == 1, 7x,x2 = 
Yrin2 Whatever the value of the other vari- 
ables; and from (17) it follows immediately 
that rrinp — wywe == 1x,x,- Hence, if the 
assumptions made are true, tests whose scores 
uncorrected for guessing have proved to be 
highly reliable need not be corrected for 
guessing. 

Equation (20) has been derived in order 
to study the dependence of the relative sizes 
of rx,x. and rp,r, on the value of p*, which 
is the ratio of the variance of the wrongs to 
the variance of the rights. The last term in 
the numerator on the right of (20) can not 
be negative because it represents the ratio of 
two reliabilities. Subtracting from it the last 
term of the denominator on the same side, 


2 2 
p I 2 p I 

(++ )-r= gr a 
TrRiRe TR 1R2 


It is easy to see here that if rr;rn. < 1, the 
term of the numerator > , — , or < the 
term in the denominator as p > , ==, or < 1. 
Continuing the assumption that raw is nega- 
tive, if the other variables on the right of (20) 
remain constant, this leads directly to the 
conclusion that whether the size of rx,x. is 
greater, equal to or smaller than that of rr,ro, 








je sP 





of the corrected scores, thus requiring the 
application of the guessing correction due to 
excessive guessing. 

The critical reader will not miss the impli- 
cation that if we make a similar comparison 
of rx,x. with rw,we by using the reciprocals 
of a and p in equations (17) and (20), the 
conclusions of this and the preceding subsec- 
tion would still stand, but with rw,w. and 
Trin. interchanged. This fact is readily dis- 
closed by the symmetry of the right-hand side 
of (16) with respect to the two reliabilities 
and with respect to the two variances. How- 
ever, the comparison of rx,x, and fw,we is of 
no practical importance as the latter has 
never been used for any purpose. 





~ 


)- (21) 

Finally it is well to observe that if each 
one of the subjects attempts all the items o! 
the tests, while dichotomous scoring is used, 
raw == —I, on = ow, and therefore, rx,x. = 
YniR2- This justifies the current practice of 
not correcting for guessing when all the sub- 
jects attempt all the items. 
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3. The Correlation of the Rights With the 
Wrongs. In order to study the effect of 
changes in Taw, it is convenient to simplify 
the notation by putting A for the last term 
of the numerator of (20) and B for the last 
term of the denominator of the same equa- 
tion. Thus (20) becomes, 


IxsX» 1 —2rewVA +4 . (22) 
TayRe 1—2rrwVB+ B 





Assuming that raw is negative, it is impor- 
tant to know now whether 








(A B 
=e >or< ¥? ’ 
r+2VA+A 1r+2VB+8B 
where 4 + Band p + 1; or whether 
I I 
= > or < a2 
I — I B 
—=+ VA + 2 VR Ve +2 
VA VB = 
But 
A 
VA " 
has a minimum value of 2 when A = 1, and 


the same applies to B or to any other posi- 
tive number. The value of this expression 
increases toward infinity both as A diminishes 
from 1 toward o, and as A increases from 7 
to. It is most convenient to consider now 
only the case when m = 2. Then B = p.? 
It was proved through equation (21) that if 
B>1,A > B> 1; and that if B < 1, 
4 < B < 1. Therefore, unless B = 1, and 
in this case rx, x» == 7r,R2, no matter what the 
value of rrw, 


(A—1)? > (B—1)?, 


which means that A is more distant from 1 
than B, whatever the signs of the differences. 





It is therefore proved that if m — 2 and 
P= #, 

2\ A 2VB ; (23) 
I1+2VA+A 1+2VB+8B 
It is deduced from this that if raw —= —r, 


the proportion of the coefficient of rpw in the 
numerator to the entire numerator of the 


= 
or? — op” 


I —_— 
2cn0w (n —1) 





"X1X2 
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right-hand side of (20) is smaller than the 
proportion of the coefficient of raw in the 
denominator to the entire denominator of the 
same equation. As rrw rises from — to o 
in (20), if the other variables remain con- 
stant, the values of both members of the 
fraction decrease because rrw is negative; 
but from inequality (23) it follows that the 
value of the numerator decreases less rapidly, 
which results in an increase in the ratio on 
the left of (20). In a similar manner it can 
be shown that as rrw passes through o and 
continues its rise toward +17, the value of the 
ratio continues to increase until it reaches its 
maximum value when rgw = +1. 


The foregoing discussion has not included 
the point of whether rx,x, > or < frire- 
The ratio is determined by the values of a 
and p as indicated in the two preceding sec- 
tions. Thus, in case that rx,x. > frre, the 
increase of rrw from —z to o exaggerates the 
discrepancy; when rx,x. < fr,r2, the same in- 
crease in rpw attenuates the discrepancy, and 
when rx,x. == 7rir2, Changes in rew do not 
affect the ratio. Then if a > 1 and p > 1, so 
that rx,x. > rrirz, a rise in raw from the 
value of —z contributes to increase the dif- 
ference and to make more necessary the use 
of the correction for guessing. 


4. The Variance of the Number of Items 
Attempted. The correlation of the rights with 
the wrongs is not usually available, and for 
this reason equation (20) is mainly of theo- 
retical importance. By using equations (3) 
the value of rrw may be expressed in terms 
of the variances of sums and differences, 
which are more readily obtained, especially 
when the tests are scored in the test scoring 
machine. Changing r and 2 for W and R, 
and changing S for T we have from (4) that 

_ 2 
Pay Cn Sahel (4a) 
4orOw 


where T —= R + W, and D = R — W;; that 
is, T is the total number of items attempted 
by one subject and D is the difference for 
each subject between his score on the rights 
and his score on the wrongs. Substituting this 
in (20), 





p—T I p—l 
+ TryRe2 +n: v Tr Re ) 





(24) 
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Bearing in mind the changed notation, it is 
evident from (3) that as rrw rises from —z 
to + 7 the value of or increases while the 
value of op decreases continuously for fixed 
values of op and ow. From the conclusions 
and assumptions of the preceding section it 
is inferred that an increase in oy produces an 
increase in the ratio on the left of (24). This 
is quite in agreement with current notions, 
according to which it is the fact that all the 
subjects do not attempt all the items that 
requires the use of the guessing correction. 
Since dichotomous scoring is assumed, o-7? is 
the variance of the number of items attempted 
as well as the variance of the number of 
items missed. But while it is true that an in- 
crease in the value of or increases rx,x. in 
relation to rp,n2 if the other variables remain 
constant, it is also true that this increase 
tends to make the two reliabilities equal if 
p <1. If p > 1, an increase in oy further in- 
creases the value of rx,x. over that of rryre, 
while if p — 1, changes in or do not affect the 
equality of rx,x, and rp,re if the assumptions 
made-are correct. It is to be noted here again 
that if all the subjects attempt all the items, 
Yaw == —1, and from (3) it is evident that 
on —= ow. Then or = 0, and as it is well 
known, the guessing correction is unnecessary. 

5. The Number of Choices. lf p = 1, 
x,:xo == ryr2, Whatever the value of n. If 
p < 1, then, returning to subsection 3, 
A < B < 1, and inequality (23) still holds 
with all its consequences. An increase in n 
would accelerate the rate at which a rising 
Tew and a rising or increase the value of 
rx,x» over that of rp,r2. But since in these 
circumstances rx,;x» < Tr r2, the effect of an 


increase in m is to diminish the difference 


between rx,x. and rpyre. 
If p > 1, B, which is equal to 


my be > , =, a0 <r. BSB D> 2, 
A > B > 1, and inequality (23) holds again. 
Since rx,x. > fr yr2, an increase in m would 
tend to exaggerate the inequality. However, 
considering that m is stated in whole num- 
bers, and that this section is dealing with 
values of m == or > 3, the cases where n = 2 
having already been treated in the preceding 


e 
2 (or, — 27R WOR; 


a 
(n — 1)’, 
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sections, it is doubtful whether n can be in- 
creased over the value of 3 to any consider. 
able extent while B remains greater tha 
or that is, when p > (n— 1). 


where 
been 5 
ances 


y 
ny, 


If B < 1 while p > 1, we know tha n= 
p—! 2 
( . + TRiR2 )> . 7 ™ bs 


but A, which is equal to the quantity on th, 
left divided by (n — 1)?, may be > or 
< s,m 


Tr R2 ‘ . 
A<sandB<A < 1. Here inequalir; 
(23) is reversed, and the effect of an increas: 
in Fw and of is now to diminish the ratio 
1x,x2 tO fryr2, While m remains large enough 
so that (25) holds true. As now rx,x, 
’ryre because p > 1, it is seen that a further 
increase in m serves to exaggerate the dis. 
crepancy in A and B and thus to accelerate 
the effect of a rising rrw and a rising o- jn 
levelling down the difference between ry. 
and fryro- 

If A or > 1 while B < 1 and p > ¢, in. 
equality (25) becomes an equation or it is re- 
versed. In case it is reversed, B << 1 < A, and 
it is necessary to know which one of the tw 
variables is more distant from 1, in order t 
know whether inequality (23) or its reversed 
form hold. However, this case is of no great 
practical importance because, as we are <ea’- 
ing with values of m = or > 3, we are very Cont 
unlikely to find a situation where inequality i 
(25) does not hold. In conclusion, it may b: 
said that, with the exception of some of the 
rare cases in which p has an exceedingly large 
value, which may upset inequality (25), a rise 
in the value of m has a tendency to equalize 
’x,x2 and frye, Which actually happens when 
nso, 

6. Determinations in Terms of the Halvs. Subs 
The split-half method enables us to make (28a 
formulas in terms of more easily available 
data for the estimation of the reliability o/ 
scores corrected for guessing, without going 
through the usual procedure. Returning to 
the notation of Section I, and preserving the 
meaning of R, W, and X in Section II, we 
have from formula (10), 


»> 





ow,* 
(n—1)* 





(n—1) 7 
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be in- where the values of o, and os in (10) have This equation is almost equivalent to (16), 
nsider- been substituted from (15). If the two vari- although the value of the reliability coeffi- 
han y. ances are equal in (3), cient is slightly higher here if Tew is nega- 
. ) (27) tive. It could have been derived directly 
w that one 20n," (1 + Trir2); F from (11) in terms of obtained scores, while 
; (16) was derived from (12) in terms of true 
or, by this and (9), scores, on account of which the latter is more 
a tie —~teen) accurate and was preferred by the writer for 
mM the Tral< . . . . 
on . . (28) the analysis made in the preceding section. 
4 From (3), 
ae ow or. -(o s+ ows) 
oe In a similar manner, SV he We Cg en 
2 ' (n—1) (n—1) 
“id . Owe 2 — TWey Wey) | (28a) Substituting from here and from (10) into 
lallty ow a 
fe) 
tees __ 4 (30), 
‘atio | — Po fae ee ve? ) . i 
Pnoug Non! — 20r,7) yo (or T ows i¢ ( ows sae) 
C ’w%— I oe , 
1X2 Xs, Xs = ! a 
further . —— OTs" Fear: (ors* + ows’ ) , ows 
: oR aes oc mee 
ne dis- : (n—1) ' (n—1)? 
elerate 2 2 2 
: I o = OTs . I . I Cws 4ow,” 
, OT In ( ee ‘ —)+ Om -—— aon: J — +o “(—— +- —— :)  — 
sa n—I ( 4on1) (n—1) “hes ' (GQ 8 (n—r}* 
X;X2 — 
2 
: I OTs 2 I I 
Ors I + )- - ow ( - :) 
> I, n- ( (n—1) os" gens * (Gon gP 
it Is re- = 
1, and P 
he tw (4or,; — ors") + —; (4ow,” — ows" ) 
rder t a ae (31) 
Xs, Xs] — f— . _— 3 
verse . a oT. CC, F Nn 
D ore; ORs —_ —— Ow, | _ 
n—vI n—vI (n—1)}* 
e dea!- —— 
re very Continuing the Spearman—Brown assump- This is a simple formula which does not 
quality tions of Section I, contain correlations or reliabilities, but it 
nay be ’, 47 nW1 OR, TW) only gives a rough estimate of rxs, Xs; 
of the —_ CnsPe in case the assumptions made are not strictly 
y large oll fulfilled. For a closer estimate, (24) can be 
aetye d and substituting the values of op, and ow, as moan » (24) 
7 a ‘ . . . us . 
qualize —— bat 
s when Tew; =—lReWs VIP / III. THe Errect or CALLING ZERO ALL 
Ry 8 RiR2 VI Twiwe-(2 é 
tise , + fwaws -(29) NEGATIVES SCORES 
Halv Substituting into (26) from (27), (28), Let X, designate a score of zero or a score 
| = (28a), and (29), above zero, X, a score below zero after cor- 
fauladie 
gfe Ons(2—PFRs Ry) 7 ae Owe ows( 2 — TW, Wey) 
- going — —lRsWs ORs : 
; 2 n—I n—I 
ing to rx... 2 — She : ( ) ( a ¥ 
ing the ows ows 
II, we On, — 27Ra Wen) Te ap 
or, 
ow ow. 
a 8 
Ors” TRs; Raj 27 RsWs ORs + > Ws, Ws] 
y - n—I (n—1)* 
i . (30) 
2 2 
: 2 Owe Ows 
Ons— 27 RsWeOR 
F : Ya—i) * tae 
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rection for guessing, and X any score, whether 
zero, above zero, or below zero. Then 


SX = 3X,+ 3X, (32) 








_ Ny 


and 


BX 4+3%% (3X. + BX)? 
x " N? »\SS 





Cx" = 


This is the complete variance of the scores 
on a test. Let ox,” represent the variance 
obtained when all the X, scores are called 
zero. Now all the 6’s vanish, and since JN, 


eG * 
TL oss? +5" (Moo? — 2MsMx) | . 
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If the measures are the sums of the score: 
on two equivalent halves of a test, al] that . 
necessary to express the relationship is to 
change the notation. Thus, 


(35a) 


However, if the measures used are the differ. 
ences between the two halves, and if these 
are assumed to have equal means and vari. 
ances, the means vanish, and 

Re eee N» 2 
TDe = OD ~~ Fy 7De . 


Substituting into (7) from (35a) and (3 


4U 





» Mp, P ‘ N, 
Ga Cp = | (os? — opp ) ie WN ( M3," omen 2MsMss) | 
T Ise y , _ 
ro “Tos 24 = (Ms,* — 2Ms.Ms,) | 


the number of cases is not reduced by calling 
zero the negative scores, 


, 3X2 (3X,)? 
=> _— , 





Cxe ™ 





in which fryise is the reliability of the test 
after all negative scores have been called 
zero. 























N N? From (7), 
and, o3° — op? = Py15 Os”, 
f » PBX? (3X)? 2X SX7 _ 
Cxe = Cx = [a7 — v2 — v2 | ° (34) 
If N, is the number of 6 scores, and, 
satan =X)? = >X,)? Tsp — Op»? = rib Osp’- 
Ny N,? Hence, 
» N N, 5 
ris os? — [rw os? +5 (Ma? — 2MoMsy) | 
— ar 2. Meee (37) 
les | os" + wy (Ms —_ 2Ms.Ms»)| 

Therefore, r.13 is the reliability of the test computed 
3X,* Nor _, , (3Xv)? from all the scores, and 7,1, is the reliability 
== 57 Pp +53]. of the scores below zero after correction for 


Substituting in (34), and letting NV, be the 
\umber of a scores, 

(3X,)? 
N? 








é : a , oy 
Cy, “COX "S11 Oo SS 


N N,? 


Nof —_ 
— ox? —[on? + (ts? —S 2M? )— 


N 
Since N= N, + Np, 


Mor . . Nee. 
TL o? +52 (My? — 2MM,)]- (35) 


COxe” = ox* — - 


guessing. The fraction on the right is a 
proper fraction and the quantity in brackets 
is positive if 7, is positive, because Ms, 





2N,.M.N»M, 
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2N, 
os M.Mp| j 
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which is the mean of the negative scores for 
the whole test is negative, while Ms,, the 
mean for the scores of zero or above is posi- 
‘ive, Granting that 7,» is positive and large, 
it may be seen in (37) that ryise < 7,18, and 
that an increase in the quantity in brackets 
will increase the difference between 7,1,- and 
,.. by diminishing 7,1... Other changes that 
will diminish rye are increases in N, the 
number of negative scores, og, and Ms,. As 
Ms, is negative, it means that the further 
below zero the location of Msp, the smaller 
rs in relation to 7,1.. In conclusion, it may 
be said that a considerable loss of reliability 
in the scores corrected for guessing may be 
expected after those scores that fall below 
zero have been eliminated, if the latter are 
many in number and spread far below zero. 


IV. THE VARIANCE OF THE ITEMS 


Kuder and Richardson’ have given the 
following formulas for estimating the reliabil- 
ity of a test in terms of the variance of the 


items or 

: ms Se, (38) 
n—TI or 

tees (39) 

n—I eo." 


in which » is the number of items, o,” is the 
test variance, pg is the average item variance, 
p is the average p and g is the average gq, 
the proportion of persons missing one item; 
and p+ q= 1. (39) is obtained from (38) 
by assuming that all the item variances are 
equal. But this never happens in test con- 
struction so that (39) is subject to an error, 
which may be of considerable size if there 
are wide differences among the item variances. 
lf the difference between the two values is D, 
subtracting the second from the first, 
2 


i gal? 4 — 9): ai 
But 

p q— pq = —Tpq Fp Tq: 

As all the p’s are not equal, r,, = —1. Also, 


=, assuming dichotomous 
Then? 


scoring. 


p qg— Pq — e,*. 
Kuder, G, F., and Richardson, M. W. “The theory of the 
wtnaties of test reliability,” Psychometrika, II (1937), 
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Since g» is positive it is evident that the value 
given by (38) is always larger than that 
given by (39), and 


——.. 


(n—1) of? 


By adding this to the right-hand side of (39), 
the value given by (38), which is the more 
nearly correct value, is obtained. 


Doing so, 














iy = r — 

S— 7 Cr (n—1)a,? 
or, 

n n (o,* — pq) 

— 
- 1 oO | (41) 
But, 

S p2 
oe, = ? — i, 


and since p + g=—=1, 
PI=p(1—p) =p— Pp’. 
Substituting these in (41), 











tp = 
eZ 
n ; (=! p) 
Le i- - ’ 
ow § oa. 
or, 
n ~p—ip’ 
ee pee) 
na— I or 


This simple formula, which can be obtained 
directly from (38), is more convenient than 
(38) for computing the value of 7,; because 
it does not involve the products. It shows 
that pq > o,”, as was to be expected in (41) 
from the fact that r,, << 1. The term pq can 
be changed in (41) without changing ay. 
This may be accomplished by adding a con- 
stant to each p. If the distribution of p’s is 
symmetrical with respect to p, it is evident 
that as the distribution of ~’s is moved up 
and down the range, ~ q varies, and reaches 
its maximum when p=.s5o. If the entire 
range of p’s is less than .50, it is possible to 
have the whole distribution of p’s either 
above or below .50. In either case p g would 
have a low value in comparison with the max- 


7 This o p should not be confused with the standard error 
of the proportion in one category of a Bernoulli distribution. 
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imum value of .25. ‘This means that, according 
to formula (41), for a given o,, and if it is 
possible for the other variables to remain 
constant, the value given by (41) is a mini- 
mum when the mean of the distribution of 
p's, which has been assumed to be symmet- 
rical, is at .50. That is, a distribution of p’s 
with a mean of .50, which is the so-called 
median difficulty of an item, produces a less 
reliable test than a distribution of p’s with 





[Vol. 9,) 


an average difficulty near 7 or near 0. 7) 

may be interpreted as being in conflict wis 
Symonds’ well known findings on the rel;. 
tion between item difficulty and reliahjjjr, 
but the point noted here is different: ang \: 
formula (41) is correct, the present , ne 
sion is unavoidable. It seems hardly ne nd 
sary to comment here that an increase in 
either o, or m will result in an increase of » 

* Symonds, P. M. ‘“‘Choice of items for a test on the 


of difficulty,” Journal of Educational Psychology, VII 
pp. 481-493, 
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ITEM VALIDITIES OF THE OTIS SELF-ADMINISTERING TESTS 
OF MENTAL ABILITY FOR A COLLEGE POPULATION 


Witiram R. Crooks 
Suffolk University 
and 
LEONARD W. FERGUSON 
University of Connecticut 


[his study was undertaken in order to pro- 
vide data concerning the difficulty and valid- 
ity of the items on the Otis test for a college 
population comparable to those which Chap- 
anis' has furnished for adults. The popula- 
tion from which the results were secured was 
the student body at the University of Con- 
necticut. All tests were administered as labo- 
ratory exercises to the students (mostly fresh- 
men and sophomores) in elementary psychol- 
ogy. A description of the sample is provided 
in Table I. The data suggest that Form B is 


Chapanis. Examination of the data in this 
table reveals the following facts: 


1. The median difference for men between 
the per cent of adults and the per cent of 
students who got each item correct is .o8 
(Range .oo to .25). 


2. The corresponding median difference 
for women is .og (Range .or to .35). 


3. The median sex difference for Chapanis’ 
data in per cent of items correct is .o4 (Range 
.00 to .23). 


TABLE I 


DESCRIPTION OF THE SAMPLE 


N* 
Form A 
ER ee eer a ae 100 
ae a 100 
Form B 
a 137 
on ERE Es SEE aE es AE See 100 


Mean Range SD Sex Diff. 
60.2 37-73 8.0 2.5.75 
57.7 40-72 ce 
55.0 35-69 i 
52.8 22-70 10.1 2.2.81 


* There is no overlap among these groups; those students who took Form A did not take Form B. 


slightly more difficult from Form A. (The 
records for Form A were randomly selected 
from 1000 test records, but records for Form 
B represent all those available at the time of 
study.) It also appears that the test is either 
slightly more difficult for women or else that 
women are somewhat inferior to men in intel- 
ligence. Since so much evidence has been 
presented in refutation of the second alterna- 
tive, we are left with only the first, if the 
differences are interpreted as significant. 

In Table II are presented data showing the 
relative difficulties (per cent of correct re- 
sponses) and relative validities (biserial cor- 
relations) of all items in Form A of the Otis 
test. Presented for comparison with our own 
hgures are those previously reported by 


Chapanis, A. “A note on the validity and difficulty of 
‘ems in Form A of the Otis Self-Administering Tests of 
Mental Ability,” Journal of Experimental Education, V 
193 pp. 246-248. 


a 
- 


4. The corresponding median for Crooks 
and Ferguson’s data is .o3 (Range .00 to .29). 

5. The median difficulty of the items on 
the basis of Chapanis’ data is .86 for women 
and .79 for men. 


6. The corresponding median on the basis 
of Crooks and Ferguson’s data is .86 for 
women and .87 for men. 


7. The median item validity on the basis 
of Chapanis’ data is .61 for women and .62 
for men. 

8. The corresponding median on the basis 
of Crooks and Ferguson’s data is .48 for 
women and .46 for men. 

g. There are (according to the writers’ 
data) 18 items for college men and 23 items 
for college women which have non-significant 
validities. Nine of these are common to both 
men and women. 


29 
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TABLE II 


VALIDITIES AND DIFFICULTIES OF THE ITEMS IN (HIGHER) FORM A OF THE OTIs Seip 
ADMINISTERING TEST FOR A COLLEGE AND AN ADULT POPULATION? 


Crooks-Ferguson Chapanis 
Men Women Men Women 
Item Per Cent Per Cent Per Cent Per Cent 
No. r Correct r Correct ¥ Correct r Correct N 
- 100 ae 100 .93 96 .99 97 
2 ~mt 96 .21 92 Pie § 89 45 83 
Bis . 26 90 .31 90 . 26 86 .38 86 
4 .25 98 mi] 97 i¥ 93 . 35 9 
5 47 93 . 06 92 .23 85 46 R88 
6 87 87 . 28 88 . 39 86 . 30 R4 
7 —.34 97 ef 95 . 50 84 47 84 
8 . 29 89 By 4 92 . 54 79 . 42 86 
9 -.13 99 P 100 . 96 99 62 99 
10 100 26 98 .74 97 51 95 
11 34 99 71 99 .3l 92 61 4 
12 —.02 98 26 98 . 68 93 88 9 
13 92 98 94 7 . 50 92 74 9) 
14 14 94 62 93 . 66 80 65 82 
15 poten 78 98 46 83 .65 80 90 68 
16 12 98 . 36 94 . 60 93 7 9 
17 12 97 46 93 .58 94 75 9 10. 
ee 50 95 18 91 .48 90 44 86 ; 
19 100 74 99 59 97 69 94 
20 26 86 22 83 . 28 87 26 92 ‘ 
21 45 7 49 74 .39 61 34 68 or 
22 44 95 45 7 65 7 56 RF We 
23 1.00 98 27 7 .29 92 53 85 hat: 
24 26 86 48 85 41 80 40 77 a 
ee a 73 .32 73 .24 65 .20 70 uty 
eee . 39 92 .10 91 . 58 94 . 56 90) 2) % 
27. za 100 47 99 7 99 .65 96 fovea 
. . 06 96 . 36 95 32 91 . 80 88 tere 
eS . 52 98 —.01 97 29 92 . 34 95 — 
30 sce . 03 97 .23 99 . 70 91 . 63 89 ites | 
31 : Ay x | 96 . 54 98 . 50 90 Pe | 86 grour 
32 : 92 98 100 . 64 97 . 58 94 ry 
33 ~ 45 86 .24 86 48 2 . 25 8] ais 
34 .19 96 .08 7 ; 100 . 90 9s stati 
35 ; 35 92 30 88 . 69 85 60 71 ten 
36 100 37 7 .49 94 52 88 vith 
/ 37 58 73 37 61 .62 62 64 52 non 
38 52 87 64 92 72 74 70 6 7 
39 ; . 29 7 . 40 88 . 62 7 . 53 76 : oy 
40 eee . 62 49 -41 24 Pe 24 .44 12 while 
ae 91 —.02 95 <a 79 . 83 76 valid 
| ee 51 90 .06 91 65 74 47 74 a 
43 __ ; 47 7 .ot 7 . 66 59 .48 44 pay 
een . 62 61 . 46 52 .70 36 . 60 34 22 1 
— 47 61 .38 47 . 60 39 . 68 27 (bast 
Ra 48 88 41 83 39 85 53 77 non- 
Me kas .48 * .38 68 . 39 68 . 51 78 whic 
' oe . 26 85 .19 62 . 52 83 .48 60 
45 70 34 70 54 56 .44 53 than 
50_.- . 56 69 . 49 79 . §2 51 . 62 44 not | 
51_- 45 71 .18 72 45 59 . 52 63 beca 
52 59 89 13 82 . 40 84 . 56 80 fon 
53... P .14 94 ome 93 . 57 90 47 89 
) 54 . 82 86 . 62 76 59 62 75 61 In 
55 42 76 .48 69 . 54 60 .27 65 Forr 
56 29 7$ 61 52 . 64 64 .65 45 quits 
57 ‘ . 29 79 . 62 91 . 63 59 te 64 Forr 
| 58 . . 55 65 ef 7 . 81 44 a 40 ; 
' 59 es .18 32 Pe 29 . 28 23 .15 1% the | 
60 a2 7 . 60 62 . 64 5A . 82 : that 


Tiititewes ae 74 . 59 45 .71 55 81 36 colle 
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ITEM VALIDITIES 231 
TABLE I]—Continued 
LIDITIES AND DIFFICULTIES OF THE ITEMS IN (HIGHER) FORM A OF THE OTIS SELF- 
ADMINISTERING TEST FOR A COLLEGE AND AN ADULT POPULATION® 
Crooks-Ferguson Chapanis 
Men Women Men Women 
Iter Per Cent Per Cent Per Cent ‘Per Cent 
\ r Correct r Correct r Correct r Correct 

) . 81 70 46 73 83 59 61 48 
f . 59 78 . 43 79 mK 68 73 61 
64 . 45 74 . 67 71 71 54 . 75 45 
F mS &9 . 55 87 . 81 64 . 85 60 
Py. a . 62 67 . 38 60 .74 46 83 31 
67 .61 44 . 59 36 . 82 28 80 20 
gR . 66 41 . 50 27 . 50 25 AT 24 
49 . 13 58 Be i | 53 . 67 43 . 69 32 
70 . 42 28 . 65 33 . 60 20 60 18 
7] Pe: 67 . 69 64 . 68 54 68 47 
79 . 62 43 45 31 . 94 34 81 21 
7 . 76 44 . 59 41 . 87 28 . 70 25 
74 .67 44 . 67 37 . 66 34 . 65 29 
75 . 60 30 . 64 15 . 78 16 80. - 12 


The data for the college population are our own, Those for the adult population are from Chapanis. 


10. There are (according to the writers’ 
ata) 30 items for college men and 22 ‘tems 
jor college women which have validities of 
50 or over. 

We may conclude from the above facts 
that: (1) there is no difference in item diffi- 
culty between adults and college students; 
2) there are no particularly striking sex dif- 
ferences in item difficulty either for college 
students or for adults; and (3) item valid- 
ities tend to be somewhat lower for a college 
group than for an unselected group of adults. 

A point of interest not brought out in the 
above discussion is the apparent existence of 
a tendency for item validity to vary inversely 
with item difficulty. Considering first the re- 
sponses of men, the median difficulty for 30 
items with validities of .50 or over is .72, 
while that for 18 items with non-significant 
validities is .95. The corresponding medians 
for women are in the first instance (based on 
22 items) .62, and in the second instance 
(based on 23 items) .92. Since, however. the 
non-valid items on any test are often those 
which are passed by more than 90% or less 
than 10% of a group, the actual relation may 
not be inverse. It may appear so here only 
because there are so many easy items and so 
few hard ones. 

In Table III are given results secured on 
Form B of the Otis Test. They appear to be 
quite similar to those already reported for 
Form A. The median sex difference is .o5; 
the median item difficulty for men is .83 and 
that for women is .77; there are 21 items for 
college men and 14 items for college women 


which have non-significant validities and 22 
items for college men and 27 items for col- 
lege women which have validities of .50 or 
over (median difficulties are .34 and .47 for 
men and women respectively). 


We agree with Chapanis that with respect 
to their validity certain of the items are mis- 
placed, but because of inconsistencies between 
Chapanis’, Otis’ and our data we are unable 
to determine any one final correct order. It is 
interesting to note that the items (59, 25, 20, 
29 and 6 of Form A) which Chapanis reports 
as the least valid items for the population he 
studied are also among the non-valid items 
of our study (for either men or women or 
both). The items (1, 34, 72, 65, and 67 of 
Form A) which he reports as most valid, 
however, are not among those which we have 
found occupying that position. 


There are a number of items which appear 
in both forms A and B, in either identical 
wording or slightly modified wording but with 
the same essential content. For men the 
median difference in the item validities of 11 
of these items having significant correlations 
is .og, and for women the median difference 
of 12 items is .11. 


We shall not attempt to summarize the 
conclusions presented above, for that would 
involve mere repetition, but we should like 
to point out what we feel to be the most im- 
portant of them. These are that the validity 
and difficulty of the items in the Otis test are 
lower for a college group than for an adult 
population. 
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TABLE III 


SELF-ADMINISTERING TEST FOR A COLLEGE POPULATION 


Men 
Per Cent 
Correct 


99 


80 


Women 
Per Cent 

r Correct 
100 
45 74 
38 91 
. 74 79 
48 77 
32 82 
.04 94 
.61 77 
.23 80 
a0 98 
. 54 97 
. 58 90 
. 06 8&3 
. 66 94 
. 06 78 
.24 94 
12 80 
87 76 
47 2 
. 40 7 
33 95 
27 92 
45 86 
62 88 
.14 92 
. 49 95 
45 71 
.16 94 
.43 74 
.35 58 
.63 65 
.42 80 
21 96 
.05 90 
. 48 60 
48 82 
23 83 
. 28 75 
. 50 62 
32 91 
. 50 75 
. 67 83 
.43 59 


Item No. 


r 


Men 

Per Cent 

Correct 
64 
51 
41 
83 
56 
70 
84 
60 
80 
68 


[Vol. 9, 
OTIs 
Women 
Per Cent 
r Correct 
. 51 4? 
. 34 36 
. 43 38 
. 15 84 
. 80 55 
. 65 48 
72 O() 
17 58 
17 73 
49 63 
35 71 
. 68 Rf) 
.47 
. 40 1M) 
56 47 
75 f 
. 86 8 
.91 84 
. 54 bo 
43 4 
62 49 
.49 35 
61 43 
59 5 
53 14 
53 3 
. 66 53 
. 65 37 
. 69 f 
66 4 
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THE PEABODY LIBRARY INFORMATION TEST: A STUDY OF 
ITS STATISTICAL VALIDITY AND RELIABILITY 


GeEoRGE H. DEER 
Louisiana State University 


Colleges and public schools alike have been 
criticized for failure to teach their pupils an 
efficient and intelligent mastery in the use of 
the library. One probable factor in this tend- 
ency to neglect so important a function in the 
oupil’s growth and development has been the 
lack of an adequate measuring device for the 
skills and information involved. Until re- 
cently no satisfactory comprehensive test had 
been devised for use in the teaching of library 
information or library use. 


The Peabody Library Information Test, 
devised by Dr. Louis Shores, head of the 
library school, and Dr. Joseph E. Moore, pro- 
fessor of educational psychology, George Pea- 
body College for Teachers, contains 188 
multiple choice, completion, and matching 
items. In its revised form the test consists of 
eight sections, each of which is really a sepa- 
rate, independent test. These consist of (1) 
The Book, 24 items; (2) Arrangement of 
Books, 24; (3) The Card Catalogue, 24; 
(4) The Dictionary, 27; (5) The Encyclo- 
pedia, 24: (6) Periodicals and Indexes, 20; 
(7) Special Reference Books, 24; and (8) 
Bibliography, 23. 


Much as such a test is needed in this field, 
none can be accepted as desirable simply 
because it happens to be available. Careful 
users of a test will insist that its worth must 
be established by its validity and its reliabil- 
ity. This study is an attempt to examine the 
validity and the reliability of the Peabody 


Library Information Test when administered 
to 1322 students in nine colleges in situations 
involving actual classroom use. After all, the 
user of a test is not concerned with what 
might be the highest coefficient of reliability 
the author of a test has been able to estab- 
lish under ideal conditions, but he is inter- 
ested in the degree of reliability which may 
usually be expected for the test under every- 
day classroom conditions. 

The 1322 subjects for this test were ob- 
tained from the freshman classes of nine col- 
leges, as follows: Bethel College, 117; David 
Lipscomb College, 143; Fisk University, 111; 
Furman University, 241; Ouachita College, 
115; Peabody College, 121; Stetson Univer- 
sity, 198; Tennessee Polytechnic Institute, 
81; Western Kentucky Teachers College, 
135; and 59 juniors from Peabody College. 
The tests were administered in the regular 
English classes. 

Table 1 shows the number of students, 
measures of central tendency, and measures 
of variability for the Freshmen of each 
college. 


VALIDITY 


This study is not concerned with the pro- 
cedures used by the authors while construct- 
ing their instrument in order to insure its 
validity; it is concerned only with the sta- 
tistical measures which indicate validity or 
non-validity for the test when used in the 
classroom. 


TABLE 1 


A COMPARISON OF THE MEASURES OF CENTRAL TENDENCY, VARIABILITY, AND RANGE FOR THE 
FRESHMAN CLASSES OF NINE COLLEGES FOR SCORES MADE ON THE 
PEABODY LIBRARY INFORMATION TEST 


College A B Cc D E F G H I 

Number Subjects _.. 117 143 111 241 115 121 198 81 135 

Median... 86.3 62.8 61.1 58.4 50. 2 31.0 63.8 54.8 76.1 
_ ee ohana 74.0 50.3 51.4 46.5 41.4 21.4 52.6 47.0 62.3 
_ RAR: 100.0 75.0 74.0 72.8 61.3 42.5 76.4 69.0 95.0 
a ae 14.5 12.4 11.3 13.2 9 10.5 11.9 11.0 16.4 
Wcanwhaindinanas 85.0 64.4 63.5 60.5 651.9 32.3 66. 0 58.3 74.8 
PE IS 17.5 18.6 17.0 17.8 15.5 15.8 18.5 16.6 21.5 
. TE 2 1.6 1.5 1.6 1.1 1.5 1.5 1.3 1.8 1.9 
RE asi 43-123 8-113 18-98 23-123 13-88 5-62 13-123 23-103 28-128 
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TABLE 2 


DIFFERENCES BETWEEN THE MEANS FOR SCORES MADE BY FRESHMAN CLASSES OF NINE 


MENTAL EDUCATION 





ON THE PEABODY LIBRARY INFORMATION TEST. ALSO CRITICAL 
RATIOS FOR THESE DIFFERENCES 


College B C D 

Dp tr BD CR DPD CR DBD 
A 20.6 9.4 22.5 9.8 24.2 12.0 38 
B 1.9 .9 8.9 2.6 12 
. 2.0 1.0 10. 
LD 8 
E 
F 
G 
H 


The first measure of validity applied to the 
test here is to determine whether it really 
discriminates between the different groups 
to which it is administered. Table 2 presents 
the differences between the means for each 
of the nine colleges and the mean of every 
other college. Differences were found to be 
as high as 52.7 between the mean for school 
A and that for school F. The column of crit- 
ical ratios is most significant in this table, 


.5 


.6 


showing that 27 of the 36 differences meet 


the general criterion of reliability, 3 sigma. 
These critical ratios run as high as to 24 for 
differences of means for schools A and F. 
This indicates that the test makes a high 
degree of discrimination between groups in 
the different colleges. 

A further test of discrimination was applied 
by comparing scores made by the freshman 
group in College F with those made by 59 
juniors in the same school who had no formal 
library instruction in order to determine 
whether the test would discriminate between 
freshmen and more advanced classes. One 
would naturally expect that in two years of 
study the college student would acquire in- 
cidentally a considerable amount of informa- 
tion about the library and its use. A test 
sensitive to different degrees of knowledge 
should be able to recognize this additional 
amount of library information acquired be- 
tween the freshman and the junior year. Re- 
sults of this comparison are presented in 
Table 3, which shows a difference of 15.6 in 
the means for the two groups, with a critical 
ratio of 6. This indicates a rather distinct 
differentiation between classes at different 
college levels. 

As a further check on the ability of the 
test to make discriminations, these scores for 
the college juniors were compared with the 
scores made by the same individuals after a 


7 


E G H ‘ 
CR D CR D CR D CR 
1 15.0 52.7 24.0 19.0 9.5 26.7 11.1 10: 
5.9 32.115.3 1.6 .8 6.1 2.5104 
6 4.8 30.2 13.8 3.5 1.7 4.2 1.712 
4.5 28.2 13.5 5.5 4.2 2.2 1.014 
19.2 9.414.1 7.1 6.4 2.7 ; 
33.7 16.8 26.0 10.7 4 ; 
6. 6 o.oo 


two weeks’ period of instruction in 

of the library, presented in the regular junior 
English classes. This two weeks of instructioy 
increased the group’s mean from 47.9 to 97. 
a difference of 49.6 with a critical rat 


TABLE 3 


MEANS, SIGMAS AND CRITICAL RATIOS FoR 
FRESHMEN AND JUNIORS IN COLLEGE | 
ON THE PEABODY LIBRARY 
INFORMATION TEST 


Freshmen Jur 

Number __ 

Mean _ 

I 1.5 
Difference between Means —_~ 
Standard Error of Difference_ 
Critical Ratio 


15.6 
26 
6.0 


20.2. This would seem to point to the tact I 
that the test was measuring, at least, what zer 
ever the instructor was teaching as library 


usage. These data are presented in Tabl 


TABLE 4 


COMPARISON OF SCORES MADE BY COLLEGE 

JUNIORS WITH SCORES MADE BY THE SAME 
INDIVIDUALS AFTER TWO WEEKS OF Se] 
LIBRARY INSTRUCTION 


Retest After er 
Initial Two Weeks det 
Test Instruction - 
Number of Subjects. 59 9 ; 
Mean Score —-.---~---- 47.9 97.5 ch 
Standard Deviations__ 16.6 25.5 oe 
Standard Error of the se 
SS ee 2.16 } 0 
Difference between Means__ 49.6 re 
Correlation, initial and final te 
RR en alas .68 dj 
Standard Error of Dai:-w2-- 2.45 : 
ee OS eo 20.2 ' 











PEABODY LIBRARY 


INFORMATION TEST 


te 


TABLE 5 


oF FRESHMEN IN THE SCHOOLS LISTED HORIZONTALLY WHO EQUAL OR EXCEED THE 
MEDIAN SCORE OF THE FRESHMEN IN EACH SCHOOL LISTED VERTICALLY 


es A B C D E F G H I 
11 9 s 3 0 14 7 31 
87 46 41 22 8 52 27 71 
89 54 44 25 9 56 38 73 
91 59 59 31 81 62 43 75 
98 71 RO 68 5 8] 63 &3 
100 97 98 96 90 99 97 99 
87 48 43 40 21 6 27 70 
97 73 69 59 40 10 74 80 
72 24 21 18 q 2 79 15 
TABLE 6 


ORRELATIONS OF THE EIGHT PARTS OF THE PEABODY LIBRARY INFORMATION TEST ON 
SCORES MADE BY 117 FRESHMEN IN COLLEGE A* 


II III IV V VI VII VIII 

I . 53 .42 3a 24 39 26 31 
I] 55 36 . 26 . 36 11 .35 
IT] .19 .15 . 25 23 yi 
I\ 27 44 22 34 
\ 34 10 10 
VI 32 , 20 
Fils ko etcmadaeenensuuncdacnanh baceeatnanan ize 

‘No coefficient has a probable error smaller than .05 nor larger than .06. 


\ commonly used measure of the degree 
{ discrimination a test possesses is the per- 
ent of overlapping from one group to an- 
ther. A fifty percent overlapping for one 
group over the median of another would indi- 
that the test was not measuring a differ- 
ence between the groups. 
lable 5 shows an overlapping ranging from 
to one hundred percent between the 
lifferent colleges. Not a single student in one 
llege ranked as high as the median of a 
certain other college and not a student in this 
second school scored as low as the median in 
the first. 
In a test made up of several parts, as this 
ne is, it is important to know whether the 
separate parts of the test are really inde- 
pendent and measuring different things; oth- 
erwise its great length cannot be justified. To 
determine this, intercorrelations were com- 
puted for all of the parts, using the students 
College A as subjects. This group was 
hosen rather than another because of their 
much higher total scores on the test. The 
oefficients in Table 6 denote relatively little 
relation between the different sections of the 
test. Each of the sections is measuring a 
different function which cannot be measured 


} 


by any one of the other parts. 


RELIABILITY 


As a measure of the reliability of the Pea- 
body Test a comparison was made of scores 
obtained on alternate items. This was done 
by correlating the scores made on the even 
items with those made on the odd items and 
then applying the Spearman—Brown correc- 
tion formula. Corrected coefficients were 
obtained as follows: Peabody Freshmen, .68; 
Peabody Juniors, .82; Peabody Juniors after 
two-weeks teaching, .96; T.P.I., .81; Oua- 
chita College, .92; Bethel College, .88. 

The probable error of estimate of a raw 
score was found to range from 1.75 to 4.5. 


SUMMARY 

This study is an analysis of the validity 
and reliability of the Peabody Library Infor- 
mation Test as administered to 1263 fresh- 
men and 59 juniors under ordinary classroom 
conditions in nine colleges, when accepted 
statistical measures of validity and of re- 
liability are applied to it. 

The findings indicate that the test is 
capable of a high degree of discrimination 
between freshmen classes of different schools 
as well as between classes at different scho- 
lastic levels within the same college. It also 








measures a distinct difference in the amount 
of knowledge possessed by a junior class 
before and after two weeks’ instruction in 
library usage. 

The eight separate sections of the test 
show such low intercorrelations as to indicate 
that each is measuring a separate function 
which is not measured by either of the other 
seven parts. 
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In only one group was the reliability ¢o. 
efficient found to be below .81, as obtained 
by correlating the even items with the odd }y 
the Pearson product moment method anq 
applying the Spearman-Brown correction 
formula. This coefficient of reliability was 
found to be as high as .g6 for one group. 

The P. E..,, for an individual’s raw score 
ranged from 1.75 to 4.5. 
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AN ANALYSIS OF THE RESULTS FROM THE FIRST ANNUAL 
EDITION OF THE NATIONAL TEACHER EXAMINATIONS 


Joun C. FLANAGAN 
Cooperative Test Service 


INTRODUCTION 


This discussion is a report of the results of 
the first annual administration of the National 
Teacher Examinations. Discussions of the 
nature of the examination service and its 
development? and of the administration of 
these examinations? have been published else- 
where. The National Committee on Teacher 
Examinations was appointed by the Amer- 
ican Council on Education at the request of 
a group of superintendents in the spring of 
1939. The Carnegie Foundation for the Ad- 
yvancement of Teaching provided funds for 
the development of the project, and the 
National Committee assigned to the Cooper- 
ative Test Service of the American Council on 
Education the task of preparing the annual 
forms of the battery of examinations to be 
used. 


The service made available to superin- 
tendents and boards of education by the 
National Committee on Teacher Examina- 


tions has three principal objectives: first, 
the provision and administration of a battery 
{ teacher examinations; second, the collation 
and distribution of information on the ap- 
praisal of other factors to be considered in 
the selection of teachers; third, the promotion 
{ research in teacher education and guidance. 

It should be stated at the outset that the 
superintendents who inaugurated this project 
were well aware that no battery of tests could 
at the present time make possible a complete 
evaluation of all factors to be included in the 
selection of teachers. However, this group of 
superintendents felt that having objective and 
comparable information on certain aspects of 
training and professional preparation would 
be of great assistance to them with their in- 
dividual problems of teacher selection. In 
common with all other persons who have been 
confronted with the task of teacher selection, 
these superintendents fully realized the diffi- 


_'An Announcement of a Teacher Examination Service, 
National Committee on Teacher Examinations of the American 
Council on Education, 15 Amsterdam Avenue, New York City, 
November 1939. 

* Ryans David G. “The Professional Examination of 
Teaching Candidates: A Report of the First Annual Admin- 
stration of the National Teacher Examinations,’’ School and 
soctety, LIL (October 5, 1940). pp. 273-284. 
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culty of interpreting the types of information 
which have previously been available to them. 

The selection of general areas to be tested 
was made chiefly through a series of confer- 
ences with superintendents and a question- 
naire sent to a larger group of superintend- 
ents in the spring of 1939. Information about 
such traits as desirable attitudes and personal 
characteristics had to be eliminated from the 
proposed battery of examinations since satis- 
factory techniques for measuring these traits 
objectively are not available. In general, the 
superintendents stated that they wished in- 
formation about the extent to which candi- 
dates could read with understanding, could 
express themselves clearly, and could handle 
numerical and non-verbal concepts _intelli- 
gently. They also wished to know to what 
extent the candidates had acquired a good 
liberal education and the extent of their 
awareness of contemporary affairs and cur- 
rent social problems. They wished informa- 
tion concerning the degree to which the candi- 
date was aware of the social implications of 
education, his knowledge of the basic prin- 
ciples of psychology as they apply to educa- 
tion, and his familiarity with the techniques 
of guiding and assisting students. Lastly, they 
wished to know the extent to which the 
teacher was really a master of those fields in 
which he proposed to teach. 


Starting with this general outline, for which 
these superintendents also supplied approxi- 
mate time allotments, the staff of the Coop- 
erative Test Service began in the summer of 
1939 the preparation of the first battery of 
examinations. 


Over a period of ten years the Coopera- 
tive Test Service has developed a method of 
test construction which has proved quite 
effective. Following this method, each step 
in the construction of the tests is performed 
by those individuals most competent for this 
particular task. The method utilizes four 
groups of individuals: first, editors in the 
various fields in which examinations are be- 
ing constructed who are experts in the subject 
matter of those fields and who have unusual 
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ability to express ideas clearly and concisely; 
second, scholars, supervisors, and other ex- 
perts and authorities who are selected from 
the entire country and who are responsible 
for the planning and final editing of the vari- 
ous examinations; third, experts in the tech- 
niques of test construction who assist the edi- 
tors and committees with the technical as- 
pects of the preparation of the examinations; 
fourth, a large group of test users who act as 
test critics and collaborators and who insure 
that the final form of the examination is 
practical as well as valid. 

The steps involved in the actual construc- 
tion of these examinations may be briefly 
outlined as follows: 


1. Developing outlines and specifications 
for the various examinations to be con- 
structed, in cooperation with a group of 
authorities and experts. 

2. Writing test items in accordance with 
the specifications and outlines, and 
editing the items. 

. Administering these preliminary items 
to students in teacher training institu- 
tions to eliminate weak and ambiguous 
items or items that are too easy or too 
difficult. 

4. Selecting and revising the items for the 
tentative final forms of the examina- 
tions. 

. Obtaining criticisms and editorial assist- 
ance from a large number of super- 
visors, heads of departments, scholars, 
experts in teacher training, and similar 
individuals. 

6. Editing on the basis of criticisms re- 
ceived. 


w 


ur 


As mentioned in the outline above the in- 
itial work on the-development of the outline 
specifications for each of the various tests was 
done by the staff editor in the particular field. 
This tentative outline was prepared after 
several weeks of careful research into what 
experts and school officials consider should be 
the relative attention given to the various 
sub-topics which are included under each of 
the major classifications into which teacher 
education is divided. Courses of study, text- 
books, journal articles, and reports by various 
professional organizations were analyzed care- 
fully, and these findings were briefly summar- 
ized for the convenience of the experts and 
authorities who were cooperating in the prepa- 


ration of the outline for the particular tes 
These materials were sent out to about 1 <>}, 
dividuals, who were asked to suggest 
in the tentative outlines or in the specifica. 
tions tentatively drawn up for the writing 
the examination questions. An effort 
made to have this group of cooperating 
viduals as representative as possible oj 
interested groups. Because of the s!} 

of time allowed for the preparation 

first form, it was impossible to make as thor. 
ough a canvass of interested groups as it js 
expected will be made with later forms 

it was considered that a fairly representatiy; 
group was obtained. On the average there 
were about eleven people who cooperated jn 
the preparation of each of the outlines. The 
total group was divided as follows: 


Teachers’ college officials ........ eae, ed 
Teacher training personnel in other col- 

leges and universities ............... 21 per cent 
Instructors in academic fields not prin- 

cipally concerned with teacher training 18 per cent 
Teachers and supervisors in city public 
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In writing the items for the various exam- 
inations, an effort was made to test for the 
ability of the candidate to apply to practical 
school situations, or, in some cases, to new 
situations of other varieties, the facts and 
principles which he had learned. In doing 
this, much use was made of the best answer 
technique; that is, the situation was described 
and several possible answers, usually five, 
were provided. The candidate was asked t 
select the answer which he regarded as best 
The preparation of such items is, of cours: 
very much more difficult than the writing 0! 
items which test the simple recognition 
facts as they have been taught; but the re- 
sultant test is generally regarded as a much 
more valid indication of the teacher’s ability 
to show good judgment in an actual teaching 
situation. The inclusion of only those items 
upon which the experts are in agreement con- 
cerning the best answer appears to be the 
most satisfactory method available at present 
of providing a test which will be fair to the 
applicants and at the same time enable th 
employing officials to discriminate among 
candidates on this basis. 

The examinations prepared in this way were 
administered to 3726 candidates in twenty- 
three examining centers on March 29-3¢ 
1940. The examination centers participating 
in this first administration of the National 
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reacher examinations were carefully selected 
» the basis of the ability of the school sys- 
‘ems to take full advantage of the examina- 
tion service. A number of the cities had pre- 
viously been using examinations as part of 
their teacher selection program, and merely 
substituted the National Teacher Examina- 


tions for the tests previously used. After ad- 


turned to the central office for scoring by 
means of the International Test Scoring ma- 
chine. The scores were reported to the school 
systems and to the candidates on April 30. 
4 general report of the administration of the 
examinations has been prepared by Dr. David 
G. Ryans.* 
RESULTS 


The most obvious finding from the admin- 
istration of these examinations is that candi- 
lates for teaching positions are not equally 
well qualified. The distributions of scores are 
shown in Table I. Inspection of the table 
makes it clear that although all of these indi- 
viduals have satisfied the legal requirements 
for certification and have accumulated the 
necessary credits, some of them know so little 
in various fields that they are able to answer 
correctly practically none of the questions in 
the examinations, whereas others make very 
nearly perfect scores on the examinations. 

Although a very large amount of evidence 
has accumulated through other testing pro- 
grams, such as the National College Sopho- 
more Testing Program sponsored by the 
\merican Council on Education and the test- 
ing program of the Teachers College Person- 
nel Association, showing that large differences 
exist between institutions, it seems worth 
while to add evidence from this particular 
testing program. The distributions of total 
scores on the common examinations for seven 
institutions from each of which there were 
more than seventy-five students who took the 
National Teacher Examinations are shown 
graphically in Figure 1. The chart shows the 
scores achieved by the students from two state 
universities, three private universities, and 
the teachers’ colleges of two different states. 
It should be emphasized that these charts are 
based on only those students from the various 
institutions who took the teacher examina- 
tions this past spring. Although these groups 
are therefore not known to be representative 


of the entire student body or even the group 
® Ibid. 
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of students prepared to be teachers by the 
particular institution, the charts show that of 
the students presenting themselves for exami- 
nation eighty-five per cent of those from one 
institution did better than did the average 
student in two of the other institutions. It is 
quite likely that some of the differences would 
be even larger were records on the entire grad- 
uating groups available for all these colleges. 
However, it should not be overlooked that 
these results indicate that there are some very 
good studies in all of these colleges. 

A second point of considerable interest 
which is brought out by an analysis of the 
results is that a fairly long and varied battery 
of examinations is necessary to describe the 
various candidates adequately. The first evi- 
dence which will be presented on this point is 
the profiles on the battery of examinations 
shown in Figure 2. This figure is reproduced 
from the profile chart which was sent to each 
individual who took the examinations with the 
suggestion that he plot his own profile on the 
chart in order to interpret more readily his 
own pattern of achievement. The ten profiles 
on the chart, representing the average scores 
made on the various common examinations by 
the group of candidates taking a particular 
subject examination, clearly emphasize the 
marked differences in pattern of these groups, 
even in cases where the combined averages 
are very similar. The profiles are self-explan- 
atory and seem to be about what one would 
expect. The Mathematics and Physical Sci- 
ence groups are outstanding on the Reasoning 
Test, the Language groups are superior on the 
English Expression Test, and the Social 
Studies, Science, and Mathematics groups are 
high on their respective sections in the Gen- 
eral Culture Test. 

Although all the various examinations in 
the battery are positively correlated, the fact 
that each one is making a unique contribution 
to the total battery is shown by the intercor- 
relations of these various parts presented in 
Table II. The smallest of the Pearson 
product-moment correlation coefficients shown 
in the table is .23 between the Fine Arts and 
the Mathematics sections of the General Cul- 
ture examination. The largest coefficient is 
.81 between the Current Social Problems and 
the History and Social Studies sections of that 
examination. The median of the fifty-five cor- 
relation coefficients in the table is .56. Pairs 
of tests which would be expected to show sub- 
stantial correlation, such as Fine Arts and 
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FicurE 1 


Tue GrapHic DISTRIBUTIONS OF THE TOTAL ScorE ON ALL THE COMMON EXAMINATIONS IN THE Nations; 
JEACHER EXAMINATIONS ACHIEVED BY THE CANDIDATES TAKING THE EXAMINATIONS IN Marcu 
Wuo Hap ReceIvep THEIR TRAINING AT CERTAIN SELECTED INSTITUTIONS 


Percen-A Private A Private A Private A State T.Col.e of T.Cole of A Privat, 
tile Unive Univ. Unive Univ. one state one state 


TY 
Unit 
VELVe 


x x x x x F 
98 


96 
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92 





79 
73 
66 


58 


42 


34 





27 


21 





16 








12 





x - x 





The percentile scale at the left is based on the returns for all the candidates taking the National Teacher 
Examinations. The middle horizontal lines show the national median, and the other two are at the 16th and 
84th percentiles of the distribution. The wide portion of each bar represents the range of scores of the middle 
half of the candidates from the particular institution. The narrow parts extend to the 16th and 84th per- 
centiles. The lines at the end extend down to the 10th percentile and up to the 90th percentile. The crosses 
represent the minimum and maximum scores for each institution. The short horizontal line near the middle 
of each bar represents the median score for the particular institution. 
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NAL INDIVIDUAL PROFILE CHART FOR NATIONAL TEACHER EXAMINATIONS 
May, 1940 
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Directions. It is suggested that each candidate fill in his VALUE or THE CHarT. This chart enables the candidate to 
own test results as reported on the card sent to him May first. appraise his own performances in various ways: 
This may be done by drawing circles around the dots cor- 1. He can compare his score on any test with the Scaled 
panting So 3s cates, = the various subtests of the National com 50, which is the score that the “average ong ll 
e , inal attery. (1.Q. 100) would be expected to make upon graduating from 
Each of the ten profiles already drawn on the chart repre- py Be college. ” ss ’ 
sents 1 average scores made on the various common exam- 2. He can compare his score on any test with the median 
ny y the group of candidates taking a particular special scores of candidates taking any particular option by means of 
ubject examination. The code for identifying these special the fil h io chest 
subject group profiles is shown below: 3 co = one _ ostes — hi in diff 
_E mer ee a ie -_ . He can study his own relative achievements in different 
4 eo oa Bag hese Sean fields by direct visual inspection of his profile, since all tests 
3. Social Studies i, ee oe are reported in Scaled Score units. This is perhaps the most 
4. Mathematics  saetlan ahaa” aati important use of the chart, since it gives the candidate direct 
5. Biological Sciences aid in appraising his own particular strengths and weaknesses 
eacher 6. Physical Sciences a ee ee It should be noted that many of the superintendents in the 
a" 7. French ee oe larger examining centers are likely to be interested in receiv- 
h and 8. German — —- - —— ing applications and transcripts from candidates whose scores 
niddle 9. Spanish —— - - - — tend to be “exceptional,” i.e., around 80 or above on the 
. 10. Latin — -- - - various examinations. 
1 per 
TOSSES COOPERATIVE TEST SERVICE ...15 AMSTERDAM AVENUE, NEW YORK, N. Y. 
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TABLE II 


INTERCORRELATONS OF SCORES ON THE TESTS OF THE 1940 TEACHER EXAMINATION BATTERY OBTAIN 
A SAMPLE OF ONE-FIFTH OF THE CASES SELECTED TO BE REPRESENTATIVE OF ALL CENYERS AN 
ALL Score Levets (N = 668) 


Test 1 2 3 4 
ee 1 45 45 A3 
_—— SE. - -73 -75 

Fema Exp.- snesea le 51 

Cur. Soc. P. en 4 
a. C.—Hist. & Soc. St.. 5 
G.C.—Literature. .--- -- 6 
G.C.—Science......- ae 
G.C.—Fine Arts_---- S 
G.C.—Mathematics __.. 9 
Prof. Information... ~~~ - 10 
Cont. Affairs._....-- 11 


M 60.43 64.37 61.76 62.44 
0 11.78 11.30 12.56 11.74 


Literature, Current Social Problems and Con- 
temporary Affairs, and English Comprehen- 
sion and English Expression turn out to do 
so, and other pairs show anticipated lack of 
agreement. The Reasoning Test, which was 
the only non-verbal test in the battery, shows 
very few substantial correlations. It is most 
closely associated with the Mathematics sec- 
tion of the General Culture Test. It is inter- 
esting to observe that the next largest cor- 
relation coefficient for this test is with the 
Professional Information Test. This is grati- 
fying since an effort was made to introduce a 
substantial factor of professional judgment 
into the questions in the latter test. 

It is of some interest to note that the evi- 
dence from these intercorrelations and from 
the profiles for the subject-matter groups 
seems to suggest three main factors included 
in the various tests. Practically all the tests 
and especially those in the fields of English, 
Foreign Languages, Social Studies, Fine Arts, 
and Professional Information show high cor- 
relations with English Comprehension. This 
general agreement of tests in these fields has 
been observed in many previous studies and 
is usually considered to be due to a general 
verbal or linguistic factor which is important 
in practically all types of academic work. A 
second general factor which is indicated by 
the results appears to include mathematics, 
science, and non-verbal reasoning. Achieve- 
ment in these fields seems to require some- 
thing more than verbal ability and this some- 
thing more is usually called a quantitative or 
reasoning factor. A third factor which is not 
quite so clear-cut as the two previously men- 
tioned is related to the Current Social Prob- 
lems, Contemporary Affairs, and Social 
Studies tests. This factor, consisting princi- 
pally as it does of an awareness of social 
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5 6 7 8 i) 1 
34 -28 46 32 7 49 
68 -76 51 -73 44 
45 67 .29 .59 40 
81 59 -58 60 7 
64 49 .58 40 
26 76 27 
37 55 
23 
62.98 63.34 60.59 63.52 62.05 62.87 
11.95 11.77 11.48 11.55 11.68 4 


problems and current happenings, m' 
what some people have called “socia| 
tivity.” 


Scaled Scores 


Although from inspection it appears that 
these three factors should account for most of 
the agreement in the scores on these tests. jt 
should also be noted that there appears to be 
a moderately large factor which is unique or 
specific to each of the tests. These specifi 
factors would be lost to a very large extent if 
the examination were greatly shortened or 
simplified. A more thorough mathematical 
analysis of the test results is planned and will 
be published in a later report. 


One other type of evidence which 
Separate testing and reporting for the 
fields is obtained from a study of indivi 
cases. In Figure 3 are shown the profi! 
the scores made by two applicants for a | 
tion teaching the Social Studies in one he 
cooperating cities. One of these canvidates 
has a very good record on the History and 
Social Studies section of the Genera] Culture 
Test and on three of the other sections of that 
test but is only slightly above the average of 
all the Social Studies applicants on the (ur- 
rent Social Problems part. The other cand 
date has a lower score on History and Social 
Studies and certain of the other sections o! 
the General Culture Test but is outstanding 
on the Current Social Problems section. 
Other differences in the profiles of these can- 
didates are apparent. One is much more 0ut- 
standing on the Professional Information | est 
than is the other. These cases have bee 
selected to show that although in gener’ 
there is substantial agreement between the 
scores achieved on various tests such as the 
Current Social Problems and the History and 
Social Studies tests, there are individual ex- 
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PROFILES OF Two CANDIDATES FOR A POSITION TEACHING THE SocIAL STUDIES IN AN EASTERN CITY 
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ceptions some of which are quite striking in 
character. 

Numerous examples of cases of this kind 
might be cited, but it should be evident from 
the intercorrelations and other data presented 
that the task of filling a particular position is 
greatly simplified if such a comprehensive 
objective profile is available in addition to 
ratings of training and experience, personality 
characteristics, and other factors considered 
important by the local group. 

There is another point in favor of a rather 
comprehensive examination that should be 
mentioned. Even if it developed that the 
forty-minute Reading Comprehension Test 
ranked the candidates in almost the same 
order as the total battery for this initial 
group, it would be very dangerous to abandon 
the use of the other tests. If it were known 
in advance that the test contained nothing 
but Reading Comprehension, there would be 
grave danger that other factors in the train- 
ing of the candidates would be slighted, so 
that in later groups the Reading Comprehen- 
sion Test would no longer provide the accu- 
rate indication of broad preparation which it 
originally did. For these reasons the National 
Committee plans to keep the battery of ex- 
aminations as comprehensive as possible and 
has taken every possible opportunity to urge 
superintendents and boards of education to 
supplement the examinations so as to include 
as many of the important characteristics of 
good teachers as possible in the selection plan. 

In addition to demonstrating that there are 
very large individual differences in the per- 
formance of candidates on these tests and also 
that the patterns of individuals whose total 
scores are at nearly the same level are fre- 
quently quite different, it seems important to 
present evidence showing as clearly as pos- 
sible what these tésts are measuring. The ex- 
periences of two of the cooperating super- 
intendents will provide a type of evidence 
which should accumulate quite rapidly as the 
tests become more widely used. The first case 
concerns the report to us by the superintend- 
ent of a Western school system that an indi- 
vidual who had taken the examinations this 
March in an Eastern center had written to 
him applying for a position. The superintend- 
ent wrote to the National Committee office 
regarding the candidate’s letter of application 
saying, “His use of the English language in 
form, in spelling, and in grammar was such 
that we had to tell him that we were not in- 
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terested in any candidate of his type.’ [p. 
spection of the records in the central office 
revealed that this particular candidate ranked 
close to the bottom among all candidates 
taking the examinations. He was in the low. 
est seventh of the distribution in both English 
Comprehension and English Expression, 

The superintendent in another city where 
the examinations were used this past spring 
talked with a number of candidates who had 
taken the examinations and were applying {or 
positions in his system. He reported to the 
National Committee that he found his judg. 
ment of these candidates substantially jr 
agreement with the results which they ob. 
tained from the National Teacher Examina- 
tions. The National Committee plans to make 
a systematic survey this fall to discover the 
relationships between various factors usually 
associated with successful teaching and the 
scores achieved on the examinations. It is 
hoped that this survey will bring out more 
clearly those elements of successful teaching 
not adequately covered by these examinations 
so that the examinations may be extended 
and so that school officials may be assisted in 
appraising those factors for which objective 
examinations are not now available. 

In the meantime it is believed that the best 
evidence concerning the validity of the exam- 
inations is contained in the items themselves. 
Therefore, a number of items are presented 
below to enable the reader to judge for him- 
self as to whether the examinations are pro- 
viding valid measures of factors concerning 
which superintendents desire information. 
These particular items have been selected for 
presentation here because they provide the 
most valid brief indication of the contents oi 
the test which is now available. In the case of 
most of these items the chances are about 
even that the average candidate will know the 
answer. But for selected persons obtaining 
scores within the top ten per cent on a par- 
ticular test, the chances are generally about 
ten to one that they will know the answer to 
any of these sample questions from the test. 
These are the items which agree best with the 
scores on the various examinations, and there- 
fore provide the best illustrations of the 
meanings of these scores. 

It is apparent that the average candidate 
applying to a superintendent for a position 
will not usually know the answers to these 
questions. On the other hand, if superintend- 
ents or boards of education wish to employ 
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individuals who can answer such questions as 
these, they may insure this by selecting those 
candidates who in addition to having the other 
desirable qualities for the position, obtained 
high scores on these examinations. In looking 
over these questions from the common exam- 
inations required of all candidates the reader 
should remember that, although the items pre- 
sented below include a very limited sample of 
the test, they illustrate the type of question 
which is known to the high-scoring individ- 
yals but is as likely as not to be unknown to 
the average applicant for a teaching position. 
It should also be remembered that, as indi- 
cated in Table I, there are quite a number of 
applicants for teaching positions in the city 
school systems included this year who know 
the answers to Aardly any of these questions. 
[hese individuals, all of whom have fulfilled 
‘he necessary requirements for teaching cer- 
tificates and many of whom have actually 
‘htained teaching positions, get a few items 
correct, partly by chance, but when their 
scores are corrected for guessing, they are 
very close to zero. 


In general the directions for these items 
ask for the selection of that one of the five 
choices given which best completes the state- 
ment. In a few cases the candidate is asked 
to find the one which is wrong. 


ENGLISH COMPREHENSION, Part II, 
It must not be supposed that the officials in charge 
of education desire the young to become educated. 
On the contrary, their problem is to impart infor- 
mation without imparting intelligence. Education 
should have two objects: first, to give definite 
knowledge; second, and more important, to create 
those mental habits which will enable people to 
acquire knowledge and form sound judgments for 
themselves. The first of these we may call infor- 
mation, the second intelligence. The utility of in- 
telligence is admitted only theoretically, not prac- 
tically: it is not desired that ordinary people 
should think for themselves, because it is felt that 
people who think for themselves cause administra- 
tive difficulties. 
2. According to the writer, a truly educated 
population is regarded by the officials in charge 


as 


1 theoretically undesirable. (23%, 2%)* 

2 the hope of democratic society. (4%, 1%) 

3 the main objective of the schools. (3%, 
1%) 

4 unnecessary. (5%, 2%) 

5 hard to manage. (65%, 94%) 
omitted (0%,0%) not reached (0%, 0% ) 


* After each answer the percentage of all candidates select- 
ing that particular choice is given; the second figure (in 
italics) is the percentage of those individuals with scores in 
the highest ten per cent who chose that answer. The correct 
answers are printed in bold-face type. 
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6. The writer evidently considers that the most 
important function of education is to 

1 impart useful information. (27%, 8%) 

2 provide vocational training. (0%, 0%) 

3 develop intelligent thinking. (69%, 
91%) 

4 create a stable social group. (3%, 1%) 

5 develop well-adjusted personalities. (1%, 
0% 


omitted (0%,0%) not reached (0%, 0%) 


Slade was a matchless marksman with a navy re- 
volver. One morning at Rocky Ridge when he was 
feeling comfortable he saw a man approaching 
who had offended him some days before. “Gentle- 
men,” said Slade, drawing, “it is a good twenty- 
yard shot—I’ll clip the third button on his coat!” 
Which he did. The bystanders admired it. And 
they all attended the funeral, too. 
7. Slade may best be described as 
1 humorous. (3%, 0%) 
2 cold-blooded. (44%, 70%) 
3 revengeful. (34%, 19%) 
4 playful. (8%,7%) 
5 daring. (11%, 3%) 
omitted (0%,1%) not reached (0%, 0%) 
8. The passage achieves its effect chiefly by means 
of 
1 suspense. (13%, 10%) 
2 exaggeration. (5%, 0%) 
3 direct quotation. (20%, 3%) 
4 matter-of-fact treatment. (60%, 
86%) 
5 detailed description. (2%, 0%) 
omitted (0%,1%) not reached (0%, 0%) 


ENGLISH COMPREHENSION, Part III, VocaBuLary, 
84. preponderant 

1 determined. (5%, 29) 

2 preparatory. (0%, 0%) 

3 advanced. (0%, 1%) 

4 thought out. (21%, 5%) 

5 predominant. (30%, 90%) 
omitted (16%,29%) not reached (28%, 
0%) 


Encuisn Expression, GRAMMATICAL USAGE, 
37. It was truly a magnificent hotel; 


~ 1 (4%, 0%) 
every one of the washstands in the various 
guest rooms were of marble, 


2 (64%, 96%) 
as were the fireplace mantels. 
3 (10%, 0%) 4 (0%, 0%) 
5 none considered wrong (18%, 49%) 
omitted (4%,0%) not reached (0%, 0%) 





ENGLISH EXPRESSION, PUNCTUATION, 
All the childrens art classes were represented in 


7 
the exhibit, and the class which had done the best 
8 

work received a prize. 

Q 

7. 1 No punctuation " on%) (10%, 1%) 
2 children’s. (56%, 84%) 
3 childrens’. (27%, 15%) 
omitted (5%,0%) not reached (2%,0%) 





8.1 No punctuation necessary. 
95%) 
2 class, (1870, 470) 
omitted (7%,1% 
9.1 No punctuation necessary. 
94%) 
work, (24%, 4%) 
omitted (9%,2%) not reached (2%, 0%) 


(73% 05 


©) not reached (2%,0%) 
(65%, 


mR 


ENGLISH EXPRESSION, CAPITALIZATION, 
“The truth is,” she continued, “we almest forgot 
to mail them.” 14 
14. 1 capital letter. (19%, 8%) 
2 small letter. (67%, 92%) 
omitted (4%,0%) not reached (10% 


ENGLISH EXPRESSION, SPELLING, 
12. 1 potatoe. (58%,94%) 


2 mattress. (2%, 0%) 
3 evidently. (1%, 0%) 
4 impromptu. (4%, 1%) 
O none wrong. (18%, 4%) 
omitted (3%,1%) not reached (14%, 0% 
GENERAL CuLiuRE, PArT I, CurRRENT SOCIA! 
PROBLEMS. 


24. Freedom of speech is guaranteed to the Amer- 
ican people by the 
unwritten law. (6%, 19%) 
penal code. (0%, 0%) 
first amendment to the Const tution. 
(63%, 98%) 
Zenger decision. », 0%) 
Declaration of Independence. (28%. 1%) 
omitted (1%,0%) not reached (0%, 0%) 
55. A governor of a state has great power in labor 
disputes through his control over 
1 the court of labor relations. (10%. 0%) 


“4 


2 the expenditure of relief funds for strikers 
(2%, 2%) 


Whee 


> 


(2% 


nm 


the issuance of injunctions. (10%,7% 

the National Guard. (56%. 91 %) 

interstate commerce laws. (10%. 0%) 

omitted (1%, 0%) not reached (1 1%. 0%) 

60. In the past fifty years, man in the Western 
world has gained most in 

health. (52%, 97%) 

native intelligence. (6%, 0%) 

emotional stability. (4%, 0%) 

security of employment. (6%, 1%) 

stability of government. (14%. 2%) 

omitted (0%,09) not reached (18%, 0%) 
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GENERAL CuLtTurE, Part II, History Social 


STUDIES, 


AND 


46. An innovation of the last ten years has been 
; the United States government’s encouragement 
of 


= 


collective bargaining. (52%, 93%) 
the regulation of interstate commerce. 
(14%, 1%) 

‘ 3 the policy of appointment to government 
positions on the basis of merit. (8%, 0%) 
cooperative action to reduce prices. (6%, 
1%) 

gee. | § cellective farming. (6%, 
: omitted (12%, 3%) not potas (2%, 0%) 


7 bh 
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GENERAL CULTURE, 





| Vol. te \ 


58. Neutrality is most often endangered 

putes arising in connection with 

1 the transport of soldiers. (2%, 1% 

radio propaganda. (1%, 0%) 
seaborne commerce. (57%, 92°, ) 
interference in the internal policy oj 
other nation. (19%, 3%) 
the public utterances of persons in hj 
position. (4%, 4%) 
omitted (6%,0%) not reached (11% 


- Wh 


an 


nm 


Part III, Lrreratvure, 

20. One of the greatest of French essavists 

Ronsard. (2%, 0%) 

Montaigne. (54%, 98%) 

Moliére. (13%, 2%) 

De alana (16%, 0% 

Villon. (1%, 0%) 

omitted (13%, 0%) not reached (1% 

70. The poet who “sounded his barbaric yay 
over the roofs of the world,” vet br 
new freedom and flexibility to poetry 

T. S. Eliot. (4%, 4%) 

Emerson. (6%, 0%) 

Edgar Lee Masters. (4%. 0%) 

Edwin Markham. (4%, 0%) 

Walt Whitman. (55%, 95%) 


omitted (21%,1%) not reached 


Ne 


nn & Ww 


WS Who = 


GENERAL CULTURE, PART IV, Screncr, 

9. Which one of the following is most 

affected by the moon? 
1 Radio communication. (2%. 0% PRO! 
Proper planting periods. (16% 
3 Fertility of animals. (2%. 0% 
+ Probability of rain. (129. 0% 
5 Sea level (62%, 100%) 
omitted (6%,0%) not reached 
The feature which especially  distinguis 
man’s nervous system from that of othe 
tebrates is the greater development in n 
the 
1 respiratory center. nes 
2 nerve sheathing. (4%, 1%) 
3 sensery organs. (10%, 1% 
4 spinal ganglia. (12%, 4% 
5 forebrain. (67%, 94%) 
omitted (5%,0%) not reached ( 

60. The physical discomfort produced by a warn 
badly ventilated room full of people is d ° 
mainly to 

1 lack of sufficient oxygen. (24%, 1 


IAC 


2 accumulated carbon dioxide. (24% 


3 rege vapors from _ people's 
1% ) 
4 oath) a water vapor. (28%, 
68% ) ; 
5 the very low humidity of the air. (12 
3%) 


omitted (0%,0%) not reached ( 


GENERAL CuLTuRE, Part V, FINE Arts, 

42. The best architecture is generally conced: 
be that in which beauty is the natural « 
sion of 

1 national feeling. (12%, 1%) 
2 function. (48%, 91%) 
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the style of the period. (13%, 1%) PROFESSIONAL INFORMATION, Part II, Cuitp Devet- 
+ strength. (13%, 6%) OPMENT AND EDUCATIONAL PsYCHOLOGY, 
5 a fine sense of decoration. (3%, 1%) 19. If a child is continually a difficult behavior 
omitted (11%,0%) not reached (0%, 0% problem, the teacher might best 


-, What modern painters are |. .ely responsible 1 refer his case toa competent author- 
r the revival of mural painting today ? ity for investigation. (71%, 90% ) 
1 English. (5%, 0%) discuss the problem with other teachers. 
J} . 7/04 (9% : 5% 
2 German. (2%, 2%) seg o) F - 
: Italian. (12%, 3%) x isolate him from the rest of the class 
pea 2%, 09%) 
4 Mexican. (53%, 91%) (27% . P 
< French. (8%, 1%) have him transferred to another class 
. {Oy /¢ C7 CF, \ 
omitted (16%,3%) not reached (4%, 0%) (270, 070) aa ae 
devise a more effective form of discipline. 
CuLtTure, Part VJ, MATHEMATICS, (16%, 57) 
Which one of the following equals 0.25% ? omitted (1%,0%) not reached (0%,0%) 


1 .0025. (53%, 95%) Six-year-old children differ most from ten- 
025. (6%, 0%) year-olds in which one of the following re- 
25. (34%, 5%) spects ? 

4 2.50. (0%, 0%) 1 Fine-muscle control. (48%, 80% ) 

< 950 om, 0% saad - 

5 25.00. (4°70, O70) : 2 Large-muscle control. (8%, 3%) 

»milte 3 Pache y vs . ~~ 
omitted (370,070) not reached (07%, 3 Visual acuity. (3%, 2%) 
The positive square root of % is 4 Auditory acuity. (1%. 0% 

1 4. (54%, 100%) 5 Metabolic rate. (20%, 13%) 

2 2. (4%, 0%) omitted (14%, 2%) .not reached (6%. 0%) 
1V2. (6%. 0%) 

+ 14, (4%, 0%) PROFESSIONAL INFORMATION, PART III, GUIDANCE AND 
=. (19%, 0%) INDIVIDUAL AND Group ANALYSIS, 


sit? IC Oo rparche oO > P . 
mitted (12%, 070) not reached (1%, 18. A boy applying for entrance to a highly sele« 


tive school attained a percentile score equiva- 

NAL INFORMATION, Part I, EDUCATION AND lent to the 67th percentile of the previous 

CIAL Po.icy, year’s entering class in that school. The school 

The part the school is permitted to play in the counselor is justified in concluding that the boy 

development of society is determined chiefly 1 probably should not be admitted to the 
the : school. (8%, 8%) 


1 attitude of the society which main- <ul admitted, would probably fail in his 
tains it. (67%, 87%) Were. (2 0,070) 

2 moral courage of the teachers. (1%, 0%) 3 if admitted, would have to work harder 

3 ability of the teachers to construct a pro- than the average entering student in order 
gram for social change. (19% 7%) to avoid failure. (30%, 179%) 

: : ey : 4 if admitted, would probably do 

4 a Se a developed in the chil- slightly better than average work. 

, %, 570 


= , ' - (36%, 70%) 
5 willingness of the school to follow political 5 if admitted, should be about 67th best in 
. a . or oO, - « ‘ : i i / . . i 
domination. (0%, 070) the entering class of one thousand. (10%, 
omitted (2%,19%o) not reached (0%, 0% 


a 
2%) 


es Niall ’ ia co ; : omitted (14%, 39%) not reached (0%. 0% 

43. Substantial modifications in the high school eave, 27) ' grit 
curriculum have been necessary since the World 
War to allow for 


. It is a basic assumption of experts in voca- 
tional guidance that 

expansion of the college preparatory cur- 1 interest in an occupation is the best evi 

riculum. (14%, 5%) dence of future success in that occupation 
the presence in high school of many (10%, 0%) 

persons with slight aptitude for 2 vocational aptitudes cannot be measured 

book learning. (61%, 88%) objectively. (4%, 2%) 

studies on a more difficult level, necessi- 3 pupils need assistance in estimating 
tated by better preparation in the grades. their own abilities, aptitudes, and 
(3%, 3%) interests. (84%, 98%) 

emphasis on cultural rather than voca- 4 there is one, and only one, occupation for 
tional subjects because of decreased em- which each individual is ideally suited. 

ployment opportunities. (5%, 2%) (0%, 0%) 


consolidation of rural high schools. (10%, 5 vocational and _ educational guidance 


2%) should be kept entirely separate. (1%, 0%) 
omitted (7%,0%) not reached (0%, 0%) omitted (1%,0%) not reached (0%, 0%) 
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PROFESSIONAL INFORMATION, Part IV, SECONDARY 
ScHooLt METHODS, 


1. The primary function of the recitation should 
be to 


give individual attention to the slower 

pupils. (1%, 0%) 

discover the pupils who have not assimi- 

lated previous lessons. (6%, 2%) 

3 discover the extent to which the individ- 
ual pupils are able to recall material read. 
(1%, 0%) 

4 develop understanding through 

group discussion. (74%, 90%) 

improve the ability of the pupils to pre- 

sent material orally. (17%,7%) 

omitted (1%,1%) not reached (0%,0%) 

38. To be successful, the “no-failure’ policy 

adopted in some schools requires 


1 the individualization of programs of 
study. (60%, 80%) 

2 decreasing attention to the superior stu- 
dents. (4%, 37%) 

3 the abandonment of school grades. (20%, 
12%) 

4 emphasis on vocational education. (4%, 
2%) 

5 elimination of pupils who cannot do sat- 
isfactory work. (4%, 2%) 
omitted (8%,1%) not reached (0%,0%) 


_ 


ht 


Mm 


SUMMARY AND CONCLUSIONS 


The above discussion gives a brief analysis 
of the results of the first edition of the 
National Teacher Examinations. Those ex- 
aminations were prepared under the direction 
of the staff of the Cooperative Test Service 
of the American Council on Education with 
the very generous cooperation and assistance 
of more than two hundred experts from all 
parts of the country. The examinations were 
designed to include those aspects of the candi- 
dates’ training and ability about which the 
superintendents desired information and for 
which satisfactory techniques of objective 
measurement were available. 

Only those items were included in the ex- 
amination which performed satisfactorily in a 
preliminary tryout and which were selected 
and agreed to by a large number of experts 
and critics including subject-matter scholars, 
teacher training experts, and selected teachers 
and supervisors in city school systems. An 
effort was made to make the examinations as 
functional and valid as possible by stressing 
the application of facts and principles and 
the candidates’ ability to exercise good judg- 
ment in selecting the best of a number of 
possible solutions of practical school problems. 

The analysis of the results indicates that 
candidates for teaching positions are not 
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equally well qualified; and that a fairly long 
and varied battery of examinations is nec.;. 
sary to describe the various candidates ade. 
quately. And finally, evidence is presented ; 
illustrate the validity of the various parts 
the examinations in measuring their res pectixe 
aspects of teacher preparation. 


In conclusion a word should be said aboy: 
the use of these examination results. The 
National Committee established examining 
centers in about twenty of the larger cities 
for the initial administration of the tests jp 
March, 1940. The National Committee has 
been interested to observe that in each of the 
more than twenty school systems requiring 
candidates to present transcripts of their rec- 
ords on the National Teacher Examinations, 
the selection procedure has also included an 
appraisal of training, experience, recommen- 
dations, and the more intangible personality 
characteristics. A further point of interest js 
that no two of the cities have adopted the 
same procedure. Some give the examinations 
a weight of about one-third, others one-half; 
some use the examinations as a preliminary 
qualifying hurdle, others require them only of 
persons who have been tentatively selected on 
the basis of preliminary interviews and rat- 
ings. But all of them require an interview 
and all seem to be making every effort to gain 
as much information as possible from persons 
with whom the candidate has studied or 
worked. A number of superinteudents have 
stated that the availability of these objective 
and comparable examination records relieves 
them of the burden of interpreting ambiguous 
records of scholastic achievement and enables 
them to focus their attention more clearly on 
the appraisal of personality characteristics. 

The National Committee on Teacher [x- 
aminations and the staff of the Cooperative 
Test Service wish to take this opportunity t 
acknowledge the very kind and generous co- 
operation of the many individuals in colleges 
and in public school systems who assisted in 
the preparation and administration of these 
examinations. Suggestions concerning the ex- 
aminations and services provided by the 
National Committee are welcomed. It is 
hoped that, through the active support o! the 
many persons interested in this project, 1! 
will be possible to make further improvements 
in the examinations and establish a larger 
number of examining centers for the con- 
venience of candidates in the coming year. 
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SOME FACTORS IN PHYSICS ACHIEVEMENT 
AT THE COLLEGE LEVEL 


D. B. Sturt and C. J. Lapp 
University of lowa 


[he prediction of scholastic success in 
various school subjects at the secondary and 
college level is one of our major personnel 
problems. Vocational guidance during this 
period consists quite largely of counseling 
with regard to the subjects which the student 
must take in order to prepare for entrance to 
a certain occupation. The student’s previous 
scholastic record suggests itself as one of the 
best predictive indices of success in various 
curricula or subjects, but there are many 
occasions when the counselor has need for 
special measuring devices to supplement the 
student’s record of scholastic attainments. 
Questions of differential prediction cannot be 
answered satisfactorily by an examination of 
school records alone. 

A related personnel problem is that of 
adjusting instructional techniques to the stu- 
dent’s particular level of ability. Granted 
that the student must take a certain subject, 
it is still possible to make adjustments in 
subject matter and techniques of teaching 
which will enable him to achieve satisfac- 
torily. A student who under ordinary circum- 
stances would be doomed to academic failure 
might conceivably do a satisfactory grade of 
work if his level of ability were to be taken 
into consideration. Likewise the student with 
high aptitude might gain a great deal more 
from a course if the subject matter proved 
sufficiently challenging. Hence, for both edu- 
cational and vocational guidance purposes, 
aptitude tests for specific subjects or groups 
of subjects should prove valuable in clinical 
counseling. 

The purpose of the present study was to 
identify certain fundamental factors related 
to scholastic success in college physics. For 
a number of years the Physics Aptitude Test 
of the Iowa Placement Examinations has 
been used at the University of Iowa to pre- 
dict the scholastic success of students enrolled 
in physics courses. It has proved itself to be 
a satisfactory instrument for this purpose, 
particularly in the sectioning of students 
according to ability. Among the other tests 





measuring factors which seemingly were re- 
lated to physics achievement, it occurred to 
the present writers that tests measuring the 
spatial relations factor might provide valuable 
data which would supplement the information 
provided by the Physics Aptitude Test. Three 
tests already available appeared to measure 
some of these abilities thought to be related 
to physics achievement, namely, the Minne- 
sota Paper Form Board, the Mann Engineer- 
ing Drawing Aptitude Test,? and the Mann 
Mutilated Cubes Test. Inasmuch as _ the 
majority of these students had taken the Iowa 
Qualifying Examination,’ which includes the 
lowa Mathematics Aptitude Test, the latter 
was included in the experimental test battery. 


Since it was administratively difficult to 
require the entire experimental group of 155 
students to take all the tests, it was broken 
up into two sections, Group I consisting of 
80 students and Group II of 75. Both groups 
were given the Iowa Physics Aptitude Test 
and the Minnesota Paper Form Board. In 
addition to this battery, Group I was given 
the Mann Engineering Drawing Aptitude 
Test and Group II took the Mann Mutilated 
Cubes Test as the third component of the 
battery. The Iowa Mathematics Aptitude 
Test was available only for those students 
who had entered the University as freshmen 
and therefore a third group of 107 students 
was set up on the basis of these available 
tests. 


During the first semester, the Cooperative 
Physics Achievement Tests (covering Me- 
chanics, Heat, and Sound) were administered 
to all the students, and the total score, con- 
sisting of the sum of the scores on the sub- 
tests, was used as the criterion of achieve- 
ment in physics. Prior to analyzing the pre- 
dictive values of the various batteries, the 


‘Only parts 1, 3, and 7 of this test were used. The re- 
maining parts do not deal with spatial relationships. 

2 The score in this examination consists of the sum of the 
we ghted raw scores in the lowa High School Content Exam- 
ination, Iowa English Training Test, Iowa Mathernatics Apti- 
tude Test, and Iowa Silent Reading Test. The average cor- 
relation of this examination with first semester grade point 
averages is .65 
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two groups were compared on the basis of 
general ability, as measured by the Iowa 
Qualifying Examination Composite Score, on 
physics aptitude as measured by the Physics 
Aptitude Test, and on physics achievement 
as measured by the Cooperative Physics 


Tests. No statistically significant differences 
were obtained. 


The zero order inter-correlations, computed 
by the product-moment method, between the 
various aptitude tests and the criterion score 
for Group I are presented in Table I. 


TABLE I 


ZERO ORDER CORRELATIONS BETWEEN APTITUDE 
MEASURES AND CRITERION FOR Group I 


N = 80 
Nis = .010 Yo, — .662 
Tn 05 ™ == .535 
Tu = 206 Yu = .160 


1. Cooperative Tests. 3. Physics Aptitude. 
2. Mann Engineering 4. Minnesota Paper 
Drawing. Form Board. 


Both the Mann Engineering Drawing Apti- 
tude Test, r,. .510, and the Physics Apti- 
tude Test, r,, = .505, tend to correlate fairly 
high with the criterion score, whereas the 
magnitude of the correlation for the Minne- 
sota Paper Form Board, r,, .258, is con- 
siderably lower. The character of the other 
intercorrelations is worthy of note, and tends 
to suggest the existence of several factors 
underlying ability in physics. Whereas both 


the Physics Aptitude Test, r,, — .662, and 
the Minnesota Paper Form Board Test. 
to, == .535, show a considerable degree of 


co-variability with the Mann Engineering 
Drawing Aptitude Test, the correlation of 
the Physics Aptitude Test with the Minne- 
sota Paper Form. Board tends to be quite 
small, r., .160. This may indicate that 
although the Minnesota Paper Form Board 
and the Mann Engineering Drawing both 
presumably measure ability to deal with 
spatial relations, the Mann Engineering 
Drawing test tends to be a better measure 
of those components related to physics apti- 
tude and achievement. 


Multiple correlations* were computed for 
the different combinations of aptitude tests 
correlated with the criterion score and these 


* These were computed by use of Griffin’s technique. See 
Griffin, Harold D.: ‘‘Simplified schemas for multinle linear 


correlation,”” Journal of Experimental Education, 1 (March. 
1933). pp. 239-254. 
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are presented below in descending order 
magnitude of R. 
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Riow = .588 1. Criterion Score. 
Riz =.557 2. Mann Engineering Drawing 
Rix =.536 3. Iowa Physics Aptitude 

La .510 4 


. Minnesota Paper Form Board 


For Group I, the best multiple correlation 
was obtained for the combination of Map, 
Engineering Drawing Aptitude, Iowa Physics 
Aptitude and Minnesota Paper Form Board 
This result seems in agreement with the pre. 
vious inferences deduced from Table I con. 
cerning the apparently different component: 
measured by the Mann Engineering Drawing 
and Physics Aptitude Tests. However, the 
engineering drawing and form board tests 4 
not add greatly to the predictive power oj 
the physics aptitude test used alone. 

As previously noted, Group IT was given 
the Mann Mutilated Cubes Test instead , 
the Mann Engineering Drawing, and in addi- 
tion, the Physics Aptitude Test and the Min- 
nesota Paper Form Board. The same type of 
correlational analysis used for Group | was 
repeated for Group II, the results being sun 
marized in Table IT. 


TABLE II 


ZERO ORDER CORRELATIONS BETWEEN APTITUD! 
MEASURES AND CRITERION FOR GROUP II 


N = 75 
Vis = .049 Yes; = .41 
fu = 664 i = 27 
ru = .261 rau = 439 


1. Cooperative Tests. 3. Physics Aptitude. 
2. Mann Mutilated 4. Minnesota Paper 
Cubes. Form Board. 


It may be noted that even under slight! 
different conditions of testing, the correlation: 
between the Minnesota Paper Form Board 


for Group II, r,, = .261, is of comparable 
magnitude to the correlation between thes 
two components for Group I, r,, == .258. the 


Mann Mutilated Cubes Test does not seem 
to show as high a degree of co-variability with 
either the criterion score or the Physics Apti- 
tude Test as did the Mann Engineering 
Drawing Aptitude Test. The correlation be- 
tween the Physics Aptitude test scores and 
Minnesota Paper Form Board is considerably 
higher for Group II than for Group I. How- 
ever, by the use of Fisher’s Z-function the 
difference was found not to be significant a! 
the 5% level. The same is true of the cor- 
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elation between the Physics Aptitude test 
ores and performance in the Cooperative 
Physics Achievement test. 
The multiple correlations for the different 
mbinations were determined and are pre- 
ented below in descending order of magni- 


R 673 1. Criterion Score. 

R..-—.670 2. Mann Mutilated Cubes Test. 
R 665 3. Iowa Physics Aptitude. 

R 376 4. Minnesota Paper Form Board. 


[he range of multiple correlations for 
Group II is much wider than in Group I, 
weesting that the combination of the Mann 
\utilated Cubes Test and Minnesota Paper 
rm Board is not as good a measure of the 
tial factors related to physics achievement 
Mann Engineering Drawing Aptitude 
nd Minnesota Paper Form Board. However, 
the physics aptitude test functioned more 
effectively as a predictive index for Group I] 
han for Group I, and as a result all the 
multiple correlations involving it are higher. 
\ small negative multiple regression coeffi- 
ient resulted in a lower multiple correlation 
efficient for four variables than for three. 
[he third group was made up of students 
m Groups I and IT who had taken all the 
Cooperative Physics Achievement Tests, and 
m there were available test data on 
the lowa Mathematics Aptitude and the 
Physics Aptitude tests. The zero order cor- 
relations are presented in Table ITT. 


TABLE III 


ZERO ORDER CORRELATiONS BETWEEN APTITUDE 
MEASURES AND CRITERION FOR Group III 
N = 107 
661 1. Cooperative Test Score. 
657 2. Physics Aptitude Score. 
779 3. Mathematics Aptitude Score. 


(he high correlation between Physics Apti- 
tude and Mathematics Aptitude is very sug- 
gestive, and tends to indicate that the two 
tests measure either the same or highly cor- 
related functions. The fact that one of the 
four subtests of the Physics Aptitude Test 
consists of problems in mathematics would, 
on subjective grounds, lead one to expect this 
result. The multiple correlation, R, .. — .699, 
which is only slightly higher than either r,. 
rr,., tends to lend further support to the 
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hypothesis that these two tests are measuring 
very nearly the same thing. 

Achievement test scores for semesters I 
and II were correlated for the two groups in 
order to determine the consistency of per- 
formance from semester to semester. For 
Group I the correlation coefficient is .774, for 
Group II it is .913, and for Group III it is 
.782, indicating a high degree of consistency 
of performance throughout the year. 

During the year following the completion 
of the research described in the preceding 
paragraphs a supplementary study was made 
of the Minnesota Paper Form Board and the 
Mechanical Movements test of Thurstone’s 
Primary Mental Abilities Test battery. One 
hundred ninety students participated in the 
study. The correlation between the form 
board scores and Cooperative Physics 
Achievement Test was found to be .152, thus 
confirming the findings of the previous study. 
The correlation for the Mechanical Move- 
ments test was .324. This was not as high as 
the correlations for the sub-tests of the 
Physics Aptitude Test as determined in an- 
other study, and consequently the test was 
not regarded by the present writers as meas- 
uring an important component of physics 
aptitude. 


CONCLUSIONS AND INTERPRETATIONS 


For the population included in this study 
or for similar populations, the following con- 
clusions appear tenable: 

1. Ability in mathematics appears to be 
more closely related than any other factor to 
achievement in college physics. While this 
cannot be regarded as an elemental compo- 
nent of physics aptitude, it appears to be 
operationally important. 

2. The spatial relations factor as meas- 
ured by the Minnesota Paper Form Board 
does not correlate highly with success in 
college physics. 

3. Understanding of mechanical move- 
ments as measured by the Thurstone Primary 
Mental Abilities Tests does not bear a close 
relationship to success in college physics as 
measured in this investigation. 

4. The Iowa Physics Aptitude Test and 
the Iowa Mathematics Aptitude Test predict 
success in college physics with a fairly high 
degree of accuracy. 
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FUNDAMENTAL PRINCIPLES IN TABULATING MACHINE 
METHODS OF STATISTICAL ANALYSIS 
E. T. JOLLIFFE 


Manager, Statistical Service, The State 
University of lowa 


FOREWORD 


One of the major problems confronting the 
research worker is that of analyzing collected 
data in preparation for final interpretation 
and presentation. The method to be used for 
the analysis of the data for any type of re- 
search project is an individual question to be 
determined by the type of problem, the ex- 
tent and form of the material at hand, the 
type of analysis to be made and the facilities 
and time available for making the analysis. 

Because of the advantages in speed, accu- 
racy and flexibility of tabulating machine 
analysis in statistical research, it is impor- 
tant that the research worker understand 
some of the fundamental principles involved 
in order that he may determine whether or 
not this method should be used, and if so, 
what preliminary steps should be taken to 
coordinate the collection of data with the 
method of analysis. Tabulating cards are 
physically movable records capable of being 
mechanically arranged, rearranged, counted, 
or tabulated, or a combination of these. Un- 
doubtedly, the most satisfactory means of 
determining the method to be used is to con- 
sult an expert in the field of research project 
analysis, one who can evaluate the various 
methods of hnalysis in terms of the factors 
governing the method. If such an expert is 
not available, the research worker must make 
his own decision and do his own planning, 


and it is for the purpose of helping him with 


these problems that the following article is 
presented. 


THE TABULATING CARD 


The tabulating card in most common use 
measures 73@ inches by 3% inches and con- 
tains eighty columns of twelve positions each 
(Illustration 1). Ten of these positions are 
the printed digits o to 9 which correspond to 
the digits of numerical data to be punched. 
Two unprinted positions are at the top of the 
card and are known as the eleventh and 
twelfth positions, commonly referred to as 


X and R. Data are recorded in this card by 
means of punched holes. 

For any project the card must be divided 
into fields to meet the requirements of that 
project. A field is a section of the card cop. 
taining an assigned number of columns where- 
in a particular type of information is always 
to appear. For example, in Figure 2 (Exam- 
ple 1), columns 41-43 are reserved as the 
field for pupil number, columns 44~45 are 
reserved as the field for score. Likewise 
grade, age, and sex are assigned to their re- 
spective fields as shown. Any field must con- 
tain enough columns to accommodate the 
maximum number of digits that will be re- 
corded in that field. Fields should be arranged 
so that punching can be accomplished with 
a minimum of effort and time. If the collec- 
tion and recording of data has been properly 
planned, the arrangement of fields on the 
card will be identical with the arrangement 
of the data from which the cards are to be 
punched. 


Any column of the card may cont uin more 
than one punched hole provided that on! 
sorting and counting operations are to be per 
formed on that column, or if it is desired t 
record alphabetical characters. Multiple 
punched columns will print from a tabulator 
only the lowest of the positions punched, ex- 
cept where provision is made for printing 
alphabetical characters. Additions can be per- 
formed only from the lowest positions 
punched. The X and R positions are usetul 
only for sorting, counting and selection pur- 
poses, except when used in combination with 
another position to form an alphabetical 
character. They can be used neither for 
addition, nor indication on a tabulator. 


CODING 
Unless it is necessary to print a record ol 
names or descriptions, all non-numerical in- 
formation should be reduced to a numerical 
code. Each type of information, represented 
by the various fields on the tabulating card, 


254 


should 
project 
such a 
tirely 

and sé 
wi yrker 


Frat 
is no} 
ing ca 
corded 

Mei 
lated 
units, 
ample 
inches 
shoul 
years 

If 
be tri 
ber 0 
be th 
serve’ 
card. 
able 
digit: 
digit: 

a de 

096. 

be fe 

latin 





INE 


ard by 
divided 
of that 
rd con- 
where- 
always 
Exam- 
as the 
45 are 
ikewise 
leir re- 
Sst con- 
te the 
be re- 
ranged 
d with 
collec- 
‘operly 
yn the 
rement 
to be 


uselu 
) pur- 
1 with 
etical 
r for 


rd of 
al in- 
erical 
ented 
card, 





arch, 1947] 


chould be coded independently. In many 
projects data sheets may be constructed in 
such a manner that coding is largely or en- 
tirely automatic, insuring greater accuracy 
and saving of time for both the research 
worker and the punch operator. 


Fractions should be avoided because there 
is no means of recording them on a tabulat- 
ing card. All fractional parts should be re- 
corded as decimal fractions. 

Measurements to be algebraically manipu- 
lated should not be recorded in compound 
ynits, but reduced to a single unit. For ex- 
ample, feet and inches should be recorded as 
inches or as feet and tenths of feet. Age 
should be recorded in months rather than 
years and months. 


If information from various sources must 
be transcribed to a final data sheet, the num- 
ber of digits recorded for each variable should 
be the same as the number of columns re- 
served for that variable on the tabulating 
card. For example, a value of 96 for a vari- 
able that may have a maximum of three 
digits should be recorded as 096, or if four 
digits are a maximum, one being reserved for 
a decimal place, it should be recorded as 
096.0. This rule, of course, can not usually 
be followed if the transcription to the tabu- 
lating card is made from the original data. 


THE Key PuNCH 


By means of the electrical key punch, data 
are transcribed to the tabulating cards in the 
form of punched holes (Illustration 2). 


The keyboard consists of twelve keys cor- 
responding to the positions in a column of 
the tabulating card. As a key is depressed a 
hole is punched in the card, and the card 
advances to the next column much the same 
as a typewriter spaces as each character is 
printed (Illustration 3). Cards are automat- 
ically fed and ejected. Information that is 
common to a number of cards can be auto- 
matically punched into each card by placing 


a punched master card in the duplicating 
rack. 


THE ALPHABETICAL PUNCH 


_ The alphabetical punch contains all the 
features of the numerical key punch, and in 
addition has a typewriter keyboard for the 
purpose of punching alphabetical characters 
(Illustration 4). 


TABULATING MACHINE METHODS 
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When one of the letters on the typewriter 
keyboard is depressed, the combination of 
holes corresponding to that letter is auto- 
matically punched into the card. This 
machine is used to punch cards from which 
descriptions or names are to be printed 
(Illustration 5). 


THE VERIFIER 

After the cards are punched, a second oper- 
ator performs the same operations on the 
verifier that have been performed on the 
punch. Instead of punching holes, however, 
the verifier checks the accuracy of the holes 
already punched. Verification in such a man- 
ner insures accurate transcription from the 
data sheets to the tabulating cards (Illustra- 
tion 6). 


THE COMPARING REPRODUCER 


The comparing reproducer is used to punch 
common information, numerical or alphabet- 
ical, into a group or groups of cards, eliminat- 
ing the necessity of a punch operator repeat- 
ing the same operation on each card of a 
group. It will also transfer punched data 
from one set of cards to another, either in 
the same field or in a different field. At the 
same time, or independently, the machine 
automatically checks its own punching. The 
operating speed is 100 cards per minute 
(Illustration 7). 


THE SORTING MACHINE 


After the cards are punched, the sorting 
machine is used to arrange them in sequence 
or divide them into groups according to some 
classification or combination of classifications 
in order to make the needed tabulations. 
The sorting operation is based on the posi- 
tion of the punched hole in the column which 
is being sorted. On the machine there are 
twelve receiving pockets corresponding to the 
twelve digits on the card. As a card passes 
through the machine, it is directed to the 
pocket corresponding to the punched hole in 
the column being sorted by means of an elec- 
trical circuit closed by a brush contact 
through the punched hole. For example, a 
card punched 8 in the column being sorted 
would be directed to the “8” pocket; another 
card punched 4 in the same column would be 
directed to the “4” pocket (Illustration 8). 

Sorting operations can be performed o: 
only one column at a time; therefore, in orde 
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to arrange a group of cards in complete 
numerical sequence with respect to any field, 
it is necessary to sort each column of that 
field. A three column field would require 
running the cards through the sorter three 
times. Within any field, sorting should begin 
with the units column and progress through 
the tens, hundreds, thousands, etc. When 
sorting extends beyond a single field it should 
progress from the minor to the major fields. 
For example, if it is desired to arrange a 
group of cards in order by age within grade, 
it would be necessary to sort first on age and 
last on grade. For a group of cards punched 
like that shown in Figure 2 (Example 1), 
the sorting sequence would be column 4g, 
column 48, column 47, and column 46. The 
machine operates at the rate of approxi- 
mately 400 cards per minute. 


THE Carp COUNTER 


The card counter is an attachment to a 
sorter which performs the functions of count- 
ing (a single column at a time) the number 
of holes punched in each position in a column, 
and the number of cards. Sorting and count- 
ing can be accomplished simultaneously on 
any column. If sorting and counting are per- 
formed simultaneously for a group of cards 
on a column which contains no multiple 
punches, the counter will record the total 
number of cards, and the number of cards in 
each pocket. Even though the sorter will 
recognize only the lowest hole in a multiple 
punched column, the card counter will count 
each hole punched regardless of the number 
of holes in that column (Illustration 9g). 


THE TABULATOR 


Printed reports are automatically prepared 
on the tabulator from the punched cards. As 
the cards pass through the machine, the 
punched holes actuate adding, subtracting, 
selecting, and printing mechanisms by means 
of electrical contacts. The arrangement of 
the compiled and printed reports is completely 
flexible. Therefore, the form of the printed 
reports is not dependent on the arrangement 
of fields on the tabulating cards. 

The data on the printed reports can be 
automatically transferred to summary cards 
simultaneously with the printing of the re- 
ports. These summary cards can then be 
used for further analysis. The speed of the 
tabulator is approximately 150 cards per 


minute, modified to a certain extent by 9) 
number of lines and totals to be prints 
(Illustration 10). 


In general there are two types of tg 
lators, numerical and alphabetical. Each wy ator 0 
perform essentially the same operations ¢y. 
cept that the numerical tabulator wil! prj; ; 
only numerical data, whereas the alphabetica! iat 
tabulator will print both numerical ap; 
alphabetical data. 


AUXILIARY MACHINES AND SPECIAL [)py; 


Because of their relative unimportance | 
gaining a basic knowledge of the principles 
of tabulating machine methods of analysis 
other auxiliary machines and specia! devices 
have not been presented. The principles 
operation of such machines are in genera! 
much the same as those described, and they 
are adapted to perform automatically most 
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problems. 


ILLUSTRATIVE PROBLEMS scor 


To give illustrative problems for al! th f th 


different types of research projects that co 
be adapted to machine tabulation would by 
impossible. But, in order that a clearer un- 
derstanding may be had of the manner in 


which some types of problems may be han- Fi 

dled, a few examples of such problems ar insV 

outlined in this section. grou 
Each sample problem begins with an illus 

tration of an original individual] record sheet 

and a tabulating card punched from this In { 


record. A list of some of the possible ta! 
lations that can be made follows, together 
with illustrative reports. 


Example 1: 

The objective test answer sheet shown in ana 
Fig. 1 has been marked by a student in 
answer to questions contained in a special 
test booklet. Fig. 2 shows a tabulating card he 
punched from this answer sheet. The upper thi 
half of the first 39 columns of the card con- pul 
tains a punched hole record of the responses 
to the first 39 items, and the lower half, a 
record of the responses to the last 39 items. 
Responses 1, 2, 3, 4, and omit, respectively, 
are in positions R, X, 0, 1, and 3 in the upper 
half of the card, and in positions 4, 5, ©, 7, 
and 9 in the lower half. Hence, the responses lov 
to items 1 and 40 (3 and 2, respectively) are sa 
punched as o and 5 in column 1; responses 
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items 2 and 41 (1, and 4, respectively) are 
ounched as R and 7 in column 2. The R in 
-ylumn 40 is duplicated to all cards and 
serves as a check for the card punch oper- 
stor in determining when the first 39 items 
have been completed. The card is returned 
to column 1 before the punching of the last 
:g items is begun. 

The following is a table showing the card 
jelds in which each of the remaining types 
{ data are punched. 


Card Columns Data Recorded 


41-43 __.._._.. Identification number 
44-45 ........ Seore 

Go caneaiece Grade 

(649 .......5 Age 


SX Se ae Sex (1 = boy; 2 = girl) 


Some of the possible tabulations are: 

item analysis; 2—frequency distribution 
of scores; 3—Sums of scores and numbers of 
pupils; 4—sums of squared scores; 5—cor- 
relation of odds and evens for reliability 
scoring to be done with card stencils). Each 
{ the above tabulations can be made in total 
or in groups by achievement, grade, sex, age. 
or responses to items or groups of items. Such 
tabulations contain the basic data needed for 
computing the desired statistical measures. 


Fig. 3 is an analysis of six items of the test 
answer sheet shown in Fig. 1, by achievement 
groups for 300 pupils. The upper, middle, and 
ower thirds are determined by grouping in 
iccordance with the total score on the test. 
In this case, approximately one third of the 
pupils scored above 33, one third between 27 
and 33, and one third below 27 (see Fig. 6). 
In order to simplify the computations the 
groups were adjusted at each extremity so 
that there were 100 cases in each group. This 
inalysis shows that a total of 128 pupils in- 
dicated response No. 1 as the answer to item 
No. 1, 73 of whom are in the upper third of 
the distribution of scores, 40 in the middle 
third, and 15 in the lower third. Also, 126 
pupils indicated response No. 4 as the answer 
to the same item, 25 of whom are in the 
upper third, 45 in the middle third and 56 in 
the lower third. Likewise, a total of 105 
pupils indicated response No. 2 as the answer 
to item 41, 29 of whom are in the upper 
third, 32 in the middle third, and 44 in the 
lower third. Item 1 and item 4o are on the 
same line because they are both punched in 
column 1, and are counted simultaneously. 


TABULATING MACHINE METHODS 
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Fig. 4 is an analysis of the same six items 
by grade. Of the 128 pupils indicating re- 
sponse No. 1 for item No. 1, 14 are in the 
eleventh grade, and 114 are in the twelfth 
grade. In terms of percentages, 23.0% of the 
eleventh graders, and 96.5% of the twelfth 
graders made this response. 


Fig. 5 is a cross-analysis between item 1 
and item 2. Of the 126 pupils selecting re- 
sponse No. 4 to item No. 1, 118 selected re- 
sponse No. 1 to item No. 2, one selected 
response No. 2, 3 selected response No. 3, 
2 selected response No. 4, and 2 omitted the 
item. Likewise, of the 278 that selected re- 
sponse No. 1 to item No. 2, 118 selected 
response No. 1 to item No. 1, 12 selected 
response No. 2, 8 selected response No. 3, 
12 selected response No. 4, and 22 omitted 
the item. 

The first column in Fig. 6 is an indication 
of the test scores in ascending order. The 
other six columns are, as labelled, the fre- 
quency and cumulative frequency for the 
boys, girls, and total group. 

Figs. 7 and 8 are sums of scores and num- 
bers of cases for each response to item 1 and 
for age respectively. The averages have been 
computed with a calculating machine. 

Fig. 9 contains the cumulative sums of 
scores by sex for each of the separate digits 
in the units and the tens positions of the score. 
The last number (5079 for boys) in the 
cumulative column is the sum of all the 
scores. The sum of the squared scores (=X*) 
is obtained by adding the cumulative sums 
for each of the digits, except zero, in the units 
position and ten times the cumulative sums 
of each of the digits, except zero, in the tens 
position. This and the remaining figures are 
obtained on a calculating machine and repre- 
sent the computations for the mean and stand 


deviation from the raw-score formulas: 
SX, /3X? — M?. - 
M =q SD: 4 a By combining 


sums of scores and sums of squared scores 
for boys and girls, a mean and standard devi- 
ation can be computed for the total group. 
Fig. 10 is similar to Fig. 9, except that each 
score has been divided into two parts—the 
score on odd items and the score on even 
items—-and cumulative sums obtained for 
each. In addition, the cumulative sums have 
been cbtained for the odd-item scores (X) 
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and each digit in the units and tens positions 
of the even-item scores (Y); so that now we 
have, by adding the column labelled “X”’ for 
the tabulation indicating digits for Y, the 
sum of the cross-products (XY) between X 
and Y. Using a calculating machine, the 
standard deviations are computed as in Fig. 
g, and the Pearson product-moment coefti- 
cient of correlation is computed from the 
raw-score formula: 


SXY — M\M, 
N 
. SD, . SD, 


Example 2: 

Fig. 11 shows two pages from a question- 
naire that is a good example of one which is 
self-coding. Parentheses indicate the codes 
and the card column numbers for each item 
of information. Fig. 12 illustrates the tabu- 
lating card punched from these two pages. 

Any type of analysis or cross-analysis of 
the information contained in the question- 
naire can be obtained from the cards. A few 
sample illustrations are shown in Figs. 13, 14, 
15. 

Fig. 13 is a distribution by age (column 1), 
and is interpreted as follows: 5 persons did 
not answer the item of age, one is from 18— 
19 years of age, 13 are from 20~21 years of 
age, 173 are from 20~24 years of age, etc. 

Fig. 14 is a distribution of items 3, 4, and 
5 (column 3), and is interpreted as follows: 
Of the 989 cases, 509 are men, and 480 are 
women; 473 are single, 469 are married, 16 
are widows or widowers, and 12 are divorced. 
The early environment of 214 cases was the 
farm, of 157 cases the small town, of 238 
cases the cities from 2000—15,000 population, 
and of 360 cases the cities over 15,000 
population. 


| Vol. ae No 


Fig. 15 is a cross-analysis of age (colump 
1) versus present population (column 7), (; 
the 173 cases from 22-24 years of age. one 
person is teaching in a city with a population 
of from 1o0o—1499, one is teaching in a city 
with a population of from 1500-1999, 4 are 
teaching in cities with populations of fro, 
2000—2999, etc. 

Example 3: 

Fig. 16 is a sample of a data sheet ; 
which has been posted a record of twelve 
measures for each individual case. Fig. 17 js 
a tabulating card punched from case No, | 
Note that numbers having fewer digits tha; 
have been allowed for in the tabulating card 
field are preceded by zero. 

From these cards can be tabulated varioy: 
distributions, sums, and progressive totals by 
digits for computing sums of squares and 
sums of cross-products for use in the formul: 
for the coefficient of correlation. Such a tab- 
ulation of progressive totals together with the 
computations for the correlation coefficient 
are shown in Fig. 18 for a sample of toc 
cases. 

The column to the extreme right of each 
series of tabulations is used for the sums of 
squares to obtain the standard deviation ot 
each variable; and all other columns are used 
for the cross-products between the column 
that is being added and the column to the 
extreme right. This tabulation follows the 
same principle as that in Fig. 9, except that 
columns for 11 cross-products are obtained 
simultaneously. Cross-products may be ob- 
tained between one variable and as many as 
20 other variables in one set of cumulative 
totals. All computations have been performed 
on a calculating machine. 


March, 
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jenceCODOSOORROORaRaeneneneneneceanneaencuceaecenoueneuecoeneneueesenenenoeons 
MOrITITETETETETITITITITITITITTTETTTT TETRA RTT E EE 
p222222222222222222222922222222222222222222222222222222222222222222222222222224 
yy9999999993939599999999999999999999999999999999999999999999999999999999995393938 
WPITITITITITITITITITITITITILITITTTTTT TTL e er TTrEnirrrrirrrr irri rrr r rrr rrr re 
copSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS 
MPTTTT 
yi) 7TPPTTIDVTIDIDUDTTTTITITATIVITIVLTIITIIITIPTIITTIIDTIAIDTTITTVITIVAIIIVIT ID 
WITT irre 


999999999999999099999900059098009990999 90099 999999980009900099989000099909989 


‘ esnnw % 1PRKMSAOTBSRNSCBRSSVSSAKEHLHKECKAV SUKHSEROTBSHHHHKBeEKC SHAN NM oenan 
«4 £08 LOCEWOCE FOR BSE UMDER PATERT 1 772.497 
Illustration 1. The Tabulating Card. 





Illustration 2. International Duplicating Key Punch. 
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avanGevcocccecenecccevccercceccccccccococcceccocsosognonssooosceneneceasesonean 


Brisas 


ae 


TT TRERRERRRTRRRRREREREEEERIEREETOTSEESECTTOTOEIPTIIO COSTE OTLOL TOOLS TIEN 
2222222222222222222222222222222222222222222222222222222222222222222222222222223 


99B999995999H95999999999939999999999999999999999999999999999999999999999I 2990005 


WUTTTITTTTET TT TLTETTETELLTT ILLITE ETL EET EL ELE citi eer rrr 


SSSHSSSPSSSHSTSSSSSSSSHHSSSSSSSSHSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS5 5555 


GEGESHEEERES OEHESSHCES EES ESSE SEESERESS dod hehe hadpetetachalpd detect Setechsist th htechchddaidebe ba L LL, 


ahhh) Te 177977 ry7222720007 1? PP PD DP EPVTIDTITIVITIIIVITTTIIVVDTVIVTIIIIIIIT IID 
ge2898 Raf §iaess BREAKER RALAL ARTHAS TRRREG LAKHS HRCRARERER RE REIERHAES 
| 995 ; 3 99959993 9999999999995 yf 
sa e ax @ e a ' “ S % *% “ee t% 2 


‘ aus Meer PATENT 


Illustration 3. Tabulating Card Punched on a Key Punch. 


international 


Ere eee eae 
LALA ALAA AR 
ee ee ed 
Fe Reeaeee e 


= 





Illustration 4. International Alphabetic Duplicating Key Punch. 
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- rst . % ? ke ee 
FS. ’ 


= 

a « 1 8 onme 

Tih s omos 

nay saennnevenvegvensneasepeceoceugeussoncnnescvccvencscncasonsogsonsenconsoavenece 
30a rPrrrvrr rit) LLERERET) PRAREERRERRERRETECIETEOESOEEELIOTEOESEOSOOSOOSEOOOOOSESIOES 
sang 9790§222222222G228222222222222222222222222222222222222222222292222222222222222249 
$5555 9999999 9G99999339999999G9999999299999999999939399993993399233399993999993393998 
6566 PPPTPTETT) PELEREETTILL) ELETETETELEL TEETER ETE TT IC TTITeRTTr irr rrrirr rrr ir 


sSSSSSSHSSSSSSSSPSSSSSSSSSHSSSSSSTSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS 
SEESSSECSERSCSOOGSPMSESS SCE ESE S CSC ESCESSESESESEOSSSCESESESSSCSECSERECEGEESSESSS 


SERRE) CARE) CRRPROREL REPRO RRPRPPRREE PREP PREER ORE RPE DROP ROPE RESEROREEEREEREEE, 
peaQeeceMereaeesesesaPsseseecseaeeesssssseceseercesssssssceecssesssseesseseeuens 


TV107 


sgoggsssssssoge $99999999 $99 SISSTIIAS IIIS Isassossssssoosesasssgsesssssssssees 
p248e ee RNS NORA RNAM AM ANSHRVHSRRBKLSHALHK SHH AHHH PHNSRVORSHHORHBeLaeRnon 
ig™ SORO LICENSEO FOR USE ROLE PATENT 1,772,662 


Illustration 5. Tabulating Card Punched on an Alphabetical Key Punch 








Illustration 6. International Motor Drive Verifier 
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Illustration 7. International Comparing Reproducing Punch. 











Illustration 8. International Horizontal Sorter. 





—————— 
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Illustration 10. International Alphabetic Accounting Machine Type 405. 





2604 








JOURNAL OF EXPERIMENTAL EDUCATION 


| Vol. 9; No. ? 











Sample: O[ J[ JC 








1OOKO 
26000 
3UWO00 
4—MOO0 
s5MOOO 
*‘OOOR 
7UMOO 
sOOWO 
sOOKO 
1loOOWO 
nMOOO 
120WO0 
1B3OKOO 
4O000K 
1I5sHOOO 
16 JMOO 
17000 
1wOOWO 
1I90WOO 
20RO00 
20NO00 
22HO00 
23RM000 
24R000 


23 OOO 
2600 XO 
27000 
28(JO0X 
2000 
30(JMO00 
3100WO0 
32MO00 
330 KOO 
34K 000 
35 MOO 
3600 XO 
37 0HOO 
38 DOOM 
39MO000 
49.0600 
410008 
4200080 
43 UOMO 
“40W00 
45OK00 
460400 
470 MOO 
48QOO00 
490800 


s0MOO00 
SOMO 
s200KMO 
3300WOO0 
54400XO0 
551000 
s6OONO 
s7UDOX 
580 00M 
ssOOOR 
eOKOO0 
618000 
626000 
680080 
+4 ORO0 
6(MO0 
66 MOD 
670 M00 
68JWOO0 
eOOWO 
770 WOO 
72WO00 
72000 
730008 
740008 


7>MODOO0 
7600 
770 WOO 
7838000 














906 


Figure 1. 


TOTITRNAT OR RYDE DIAZBDATAF wenrramrans 
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pecesecononeseeeosoooescoecess 


TETEIEETEELITTTEREETEEEETT EET 
22222222222222222222222222222222222222 2 2 2)2 2}2 42 222222222222222222222222222224 


923332233339339233923333333333333332333333 93 : Seiya hat «soe Lae em peeaabe 





PYUTETITITETETTTETeTTererTirey’ 


SSSISSSSSSSSSSSSSSSSSSSSSSS5SS 





np hchoct pple etal 


2b PERRRRAEROAAE) C1 | | PARREROEREREE | OB 
PEQRURPERES ERLE SRR ERER TAREE eoRARRe EES 


7777777777777 pyaar aI 
SHSRRCRAKEETERRR ESR SEREreEeEE 
~~ 


pengesgsesegssacaseggcoseaaeesssessges TESESEIEPIOIEEIIIIririrriiiyy | @ 
CLE EHRHN ROMERO BANRHEARAKBAYVARABRHOVKRSBEYS Vevessee seeent@eouecanvans a” a” 
am 5OR0 FOR USE WHPF® PATENT (212.492 
Figure 2. 
Response Response 
Item ] 2 3 4 Omit Item 1 2 3 4 Omit Total 
Group 
Upper 73 1 25 1 40 71 2 14 9 4 100 
Middle 40 5 4 45 6 56 7 22 7 8 100 
Lower 15 8 4 56 17 42 7 24 13 14 100 
Total 128 13 9 126 24 169 16 60 29 26 300 
2 Upper 95 3 1 Dawe 3 29 10 55 3 100 
Middle 90 3 4 2 1 5 32 13 41 9 100 
Lower oe a eee 8 44 14 27 7 100 
Total 278 6 10 3 3 16 =105 37 =123 19 300 
3 Upper 58 15 13 3 11 42 54 2 14 25 5 100 
Middle 32 11 11 9 37 32 3 21 32 12 100 
Lower 17 14 6 13 50 18 7 21 36 18 100 
Total 107 40 30 25 98 104 12 56 93 35 300 


Figure 3. 


Response Response 
Item 1 2 3 4 Omit Item 1 2 3 4 Omit Total 
Grade 
1 11 14 eee 31 12 40 28 7 13 6 7 61 
12 114 9 9 95 12 141 9 47 23 19 239 
Total 128 13 9 126 24 169 16 60 29 26 300 
2 11 57 Demme 2 1 41 2 27 10 18 4 61 
12 221 5 10 1 2 14 78 27 105 15 239 
Total 278 6 10 3 3 16 105 37 §=123 19 300 
3 11 22 7 5 2 25 42 17 2 8 20 14 61 
12 85 33 25 23 73 87 10 48 73 21 239 
Total 107 40 30 25 98 104 12 56 93 35 300 
Figure 4. 


, —_—s.) TT ADTITT ATWIATS* Ueascvryrrvyarrm wrrwrrnnre 
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ITEM 2 
Response 1 2 3 4 
l 118 4 5 1 
2 - ees l 
3 8 ae 1 = 
4 118 1 3 2 
Omit 22 1 i —_— 
Total 278 6 10 3 
Figure 5 
Boys Girls 
Score Ik C.F y <€.: ¥. 
1 era eee ae 1 1 
15 1 1 Pian 1 
_ eee 1 2 3 
19 3 4 3 6 
20 3 7 3 9 
21 1 8 4 13 
22 4 12 9 22 
23 3 15 10 32 
24 5 20 5 37 
25 7 27 9 46 
26 8 35 12 58 
27 7 42 5 63 
28 9 51 10 73 
29 3 54 7 80 
30 6 60 9 89 
31 8 68 8 97 
32 12 80 8 105 
33 7 87 1 106 
; 34 6 9 7 113 
a) 35 8 101 4 117 
a 36 3 104 3 120 
: 37 2 106 5 125 
38 10 116 4 129 
39 4 120 1 130 
40 3 123 z 132 
41 4 127 4 136 
| ae: 127 3 139 
43 1 128 1 140 
44 3 131 1 141 
45 6 137 2 143 
bad 46 — Dee xsucen 143 
; 47 a ewer 143 
' } 48 1 . es 143 
i 50 1 142 1 144 
: 51 1 143 2 146 
if 52 1 144 1 147 
+ | 54 . ° ee Sawees 147 
) ae 55 .- We wiksawe 147 
Bee. | 56 1 147 1 148 
me | 59 . 10 cageeke 148 
1 61 1 149 __.__. 148 
het) .— eee 149 149 
cd 66 S Se dawn 149 
pai] 73 ae 149 
| : Figure 6. 
] 
ae 
eh 
514 
bat 
aa 
; 
268 
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Omit Total 


128 
13 
9 
2 126 
1 24 
3 300 
Total 
r.6 6. F. 
1 1 
1 2 
2 4 
6 10 
6 16 
5 21 
13 34 
13 47 
10 57 
16 73 
20 93 
12 105 
19 124 
10 =134 
15 149 
16 165 
20 3185 
8 193 
13 206 
12 218 
6 224 
7 3a 


DOR CODD HOR IDO ONO 
bo 
fo 2) 
_ 


Sum of No. of 


Scores Cases 
4746 128 
845 13 
259 9 
3610 126 
610 24 
Figure 7. 


Sum of No. of 


Scores Cases 
697 23 
23 1 
267 9 
2021 62 
4351 134 
1749 56 
387 13 
75 2 
Figure 8. 
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Average 


30.3 


23. 


29 
32 
32 
31 
29 


37 


[Vol. 9, No, ; 


0 


rwormeumn- 


v 





Mar 


Me 






































































































































ol.9,No; March, 1941] TABULATING MACHINE METHODS 267 
sje cum. sum lele ooo Even 
wit OF SCORES ju) ® SCORES SCOWES 
ad Ne —— ki (x) 

| 
™ ) 359 ic 364 300 
Boys B 1039 : 777 4 
7 1349 17 203% 112¢ 
6 i679 16 147 <¢ 1462 
5 2 6 74 5 1 7 69 19 
“ 31864 4 2147 229 
3 3600 | 3] ‘ ‘S : 
2 4124 | |2] 5146 “ 
1 4669 | ir 357 4 
5079 ct WE wy 
| N= 300 161 
7 To 3 | 1 96 2 
Averag, 6| | pe Ia} | 1194 132 
es 5} | 577 I 5A 5 14 ' 
oe . 4) | 1493 ZT 64 w+: 
os 3 3738 t 
os 2 5007 Dy- or Exy | | | 6 7.3/9 Jo bbs 
<5. 1| 5079 
25. 4 SOFS 2151 scttlasiccnal 23/. 0633 252.467 
™ /#. 7 $00 
=x ASK SET $D | | 535% 
= e/n 4/A22./656 on.* 40 
Mm 33-6354 I 29.7477 _ ———-— 
So 7.528 A |_| 7 3¢ 
ry 
| 
| | 
Average 
30.3 ‘ : 
23.0 9] $4 
9 - te] { 
— 7| 1167 
> lo 6] 1389 
32.5 GIRLS 9) 299 le | pte 
31.9 a| 731 [5 | 54 
29 8 7| 1085 | |4 227 
97 : 6| 1561 3 he 4 
5 2081 2 $3 : 
a 2483 1 of 4 
3 2789 . 
2 3433 
la 4031 3 126 
4491 > 1060 
4 4 > 
6 65 41 1 
s| 2a8 t 
4) 872 Z x* 67 SIS 
2541 7 
3 4388 2/n 228.000 
1 4491 m r: % 9367 
| 4491 wW+=149 
SD F.SS1F 
= x S¢ S313 
=n PTF. ASFo 
™m 30. /409 
3.0 $./720 Figure 10. 
Figure 9. 
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— 
Applies to all. Indicate with a the 3. Applies to all. Indic . 
oe ae IR og we pPrnvack. Benen S hs A. pplies to a ndicate with a cross (x) your sex: | 
to your nearest birthday: (0) > =" 
(1) ~~........female | 
(B)...........18 to 19 years | 


(X).-.......20 to 21 years 4 


Applies to all. Indicate with a cross (x) your soda) 
etatus: 


(0) ...........22 to 24 years 
(2) —......single 
(1) ~......25 to 27 years 
, (3) ~-X.married 


| 
(2) A 28 to 30 years | 
| 





(4) ..—..... widow or widower 
COD. ccsretrnns 31 to 33 years 
(5) ..........divorced 
(4) .........34 to 36 years 
5. Applies to all. Indicate with a cross (x 
(5) ~..........37 to 40 years yo... marci : © (2) preny | 
(6) weense41 to 45 years (6) —%...farm 
7 46 to 50 years (7) .-.--2-0---8mall town (under 2000) | 
(8) ........51 to 60 years (8) —.....city (2000 to 15000 pop.) 
(1) .2-<=- (9) ....---61 years and over (3) a (9) --......city (over 15000 pop.) | | 
} 
} 
6. Applies to all. Indicate with a cross (x) your re | 
ligious preference: 


2. Applies to all. Indicate with a cross (x) the age 
j group below which best represents your age at the 


— . 
time you entered the teaching profession: (*) mane 


i (1) —-A._.Methodist 


(R).........15 to 17 years } 
 —— Baptist | 
| 
| 





| 
(0) ——....--Presbyterian | 
| 
| 
| 
| 


7 | 18 to 19 years 


(3) ..-0—--Lutheran 
(0) ......-...20 to 21 years 


a) ~—— (4) ...........Congregtstional | 
K.....22 24 years 

(5) 
ne (2) —....-.25 to 27 years 


ie (6) ..........Other Protestant sects 
Pit.4 (3) ....-.-.-28 to 30 years 


~Episcopalian 





(7) 2.-------J ewish 
(4) —......31 to 33 years ) 


~= 


; : . *-* 
7. Applies to all. If you have no religious preference 
(6) f -34 to 36 years put a cross (x) in the space below: ; 
© plete (8) ......n0 preference 


(7) ~.-..41 to 45 years 


i: 
an 8 Applies to all. If you are a member of the sect indl- 
& (8) 46 to 50 years cated above, put a cross (x) in the blank below: 
4 ' (2). (9) —~-....51 years and over (4) carssssees(9) ~Dchareh member 
ae | é 

2 

a 
’ 


<< oe 














Figure 11. (1st page) 
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| 9. Applies to all. In the list below, indicate with a iL. 
cross (x) your father’s occupation while living: 
Note: Check but one item. 


(EB ).K_.tarmer, agriculturist 
(X).........agent, office worker, salesman, realtor 
(0) ........-banker, financial agent 


(2) accrecesees contractor, building trades: 


carpenter, 
bricklayer, plumber, ete. : 


(8) cena government and public service 
(3) —......merchant, dealer, proprietor 


(4) .........professions: doctor, lawyer, dentist, 
teacher, preacher, etc. 


(5) .......railroad employee 


(6) ...........mining (laborer) 


(7) ....---.trades other than building, (skilled me 
chanic, painting, decorator, ete.) 
 ) manufacturer, public utilities 
' 
(B) axnevvveeee-(9) o---e---0--tnskilled laborer 


10. Applies to all. In the list below, indicate with a 
cross (x) the type of teaching certificate you now 
have registered with your county superintendent: 
(If you hold more than one certificate, indicate only 
the highest.) 


(R/0)...%....First Grade State 
(R/1)...........8econd Grade State 
(B/2) -enssscsoee Third Grade State 


12. 


14. Applies to all except rural teachers. 


Applies to all. If the certificate which you indicated 
in the preceding list was granted you by examination, 
place a cross (x) below: 


(8) _X. certificate granted by examination 


Applies only to holders of Wife certificates. If the 
certificate you have checked in the preceding list is 
validated for life, place a cross (x) below: 


(6)......—. (9) ........life certificate 


13. Applies only to holders of special certificates. If you 


have indicated a special certificate, write below the 
name of the special subject you are certified to teach: 





: If you are not 
teaching in a rural school, place a cross (x) before 
the population group to which the city in which you 
are now teaching belongs: 











(R ).-.......below 500 
(X) .nnnean-e.500-999 

(0) eeenenee-1000-1499 
(1) seeveaneee..1500-1999 
(2) ....AK....2000-2999 
(3) snecsoseeee-3000-3999 
(4) ~..........4000-4999 
(5) wenem--5000-9999 
(8) <enss-ae---10000-14999 
(7) avono-ae~-15000-24999 
(8) -..-....25000-49999 


(7) snmevesnen (9) snneesen-50000 and over 


(R/3)............Fourth Grade State 






(R/4)............Rural School Teacher’s State 
| a Special State (Kindergarten) 
(RB /6)............Special State (Primary) 
(R/7)..........-Other Special State 
6 First Grade County Uniform 
(X/1)-~-~......Second Grade County Uniform 
+S ; Third Grade County Uniform 


(X/2)........---Special County 

8 | Provisional County 
(X/5).........High School Norma) Training 
(X/6).........Life Diploma 


The above list should contain only one check mark. 





15. Applies to all who have taught in more than one com- 


munity. If you have taught in a town or city other 
than the one in which you are now teaching, indicate 
with a cross (x) the population group to which the 
city belongs in which you taught just before coming 
to your present position: 












(R).........below 500 
(X).. ..500-999 

(0) .....-;---1000-1499 
(1) ..2&..1500-1999 
(3) ... .2000-2999 
(3) .3000-3999 
(4) . 4000-4999 
(5) ... 5000-9999 
(6) ~........10000-14999 
(7) ——.....15000-24999 


...25000-49999 


| ae 
(8)... (9) .....---50000 and over 





Figure 11. (2nd page) 


March, 194! 


TABULATING MACHINE METHODS 


271 








or 


* 





a 


= 
ines Se 





Tat as ee sea 


3 Po 


ett 
ee 


oa? 


— 


a ee 
a gn eins oa 
ee ae 








270 JOURNAL OF EXPERIMENTAL EDUCATION 


[Vol. 9, No.3 
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peneensveny 


ah 
HSSSSS555555 


} 98599995959 SH9HISFAIGI999G44 
RS 


Figure 12. 


Unans. R xX 0 1 2 3 a 5 6 7 8 9 Total 


Age 
(Col.1) 5 1 13 173 182 181 181 += 105 75 37 31 39 16 989 


0 1 2 3 4 5 6 Total 
ec .-.-.-.--. 39 480 4738 469 16 12 214 157 238 360 989 


~_ 
« 
i) 


Col. 3 


Figure 14. 


Present Enrollment (Col. 7) 
Unans. R x q 1 2 3 4 5 6 7 8 9 Total 


Nn erie ee a. ee ee 1 1 a en 
OR ee oi akatncns Sy a eevee ae ee ye Sp as 1 
xX 1 i a 6 3 eee 1 13 
0 1 : l 1 4 2 7 54 36 10 21 36 =:173 
1 ae eae 7 46 35 13 21 56 =: 182 
2 7 ae Rete 2 4 3 36 36 15 26 57 = 181 
3 2 ee Sear 1 5 6 24 22 12 17 41 131 
eer re en nee 1 5 18 20 6 20 33 =: 105 
5 Be Seis aw. ae eacatia tis alee eee 2 1 1 13 11 2 18 26 7 
a hoe oral ne, ee NI a i a i 3 2 6 1 5 20 37 
a he eee a ae ee 3 8 1 8 11 31 
Dt aiteeuaO auras a aES ables eee week OS goed 3 4 4 2 4 21 39 
a a a ca Ae a ae ae ea oa 4 2 1 5 16 
Total 8 - Se 2 2 10 17 36 211 185 64 145 307 989 
Age (Col. 1) 
Figure 15 
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CAIE 
No 


DP RPP PPP PPR re 
CODAIAMFWdDE COD AQMNkWDE 
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as 
St / e341 s . 7 8 9 se Wf 12 
ie 1 78 72 90 88 090 87 82 74 80 46 083 60 
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CORRELATION COEFFICIENTS AND OTHER CONSTANTS 
FOR COMBINED AND SEPARATED DISTRIBUTIONS 


Douctas E. Scates 
Duke University 


There are numerous occasions when one 
wishes to obtain correlation coefficients and 
other series constants from several populations 
combined into one, or from a breakdown of 
a general population into two or more sub- 
populations. It is somewhat arduous to make 
new tabulations of the combined or separated 
populations. There are methods by which any 
number of tabulated distributions may be 
combined without additional tabulation or un- 
due labor of calculation, and by which any 
tabulated distribution may be separated out 
from a compound table without further tabu- 
lation. The methods are equally applicable to 
calculations based on paired series which have 
not been tabulated, but for convenience of 
expression series will be spoken of as though 
tabulated. 


A method of accomplishing this result was 
published some time ago by Dunlap,’ and 
his method is calied for when only means, 
standard deviations, and correlation coeffi- 
cients are available. When however one has 
access to correlation work sheets the method 
herein presented? will probably be found more 
facile—especially when more than two sub- 
groups are to be combined or separated. It is 
also highly flexible when certain special pur- 
poses apart from a simple combination are to 
be accomplished. 


The method herein described is simply to 
sum, across the various groups to be com- 
bined, the respective totals which are normally 
required for the calculation of the desired 
constant in any single group, and then employ 
these sums in the formula in the usual way. 
For example, in the case of correlation for a 
single group, one commonly obtains six sum- 
mations: N, =x’y’, =x”, =x’, Sy’*, and Sy’— 
the primes indicating that an arbitrary origin 
has been used. To calculate the correlation 


‘Dunlap, Jack W. “Combinative Properties of Correlation 
Coefficients,” Journal of Experimental Education, V (March, 
1937), pp. 286-88. 

*This procedure was worked out and described in the 
writer's doctoral dissertation: Selective Admission and Selec- 
tive Retention of College Students at the University of Chi- 


yy p. 107-30. Doctor’s thesis. University of Chicago. 1926. 
: Pp. 


for two or more groups combined (as though 
tabulated together on the same table), one 
needs only to combine the N’s for the two 
or more groups, then combine the &x’y’ values 
for the two or more groups, and so on through 
the six summations. It is assumed only that 
the same working origins have been used for 
the different tables, and that the class inter- 
vals are the same; if they are not, corrections 
can be applied which will be described later. 
These combined summations are then used in 
the correlation formula just as though they 
had been derived directly from the compound 
table consisting of all the subgroups tabulated 
into one—for they will be the same values 
that one would get by actually combining the 
tabulations into one. If it is a compound 
mean or standard deviation that one wishes 
instead of a correlation, he combines only the 
summations which are needed, and substitutes 
them in the appropriate formula. 


The procedure holds equally well for the 
correlation formula based on differences, 
x(x’ — y’)*, instead of products, =x’y’: one 
sums the appropriate totals for that formula. 
In general, any form of correlation formula 
may be employed, so long as the working 
origins and class intervals are kept uniform, 
or the differences corrected for. Formulas 
which call directly for deviations from the 
actual mean of each subgroup do not keep the 
origins constant (unless all the means happen 
to be the same) and hence call for a correc- 
tion appropriate to differing origins, as ex- 
plained later. 


The rationalization of the procedure may be 
set forth as follows: if several separate tables 
which are to be combined were actually com- 
bined by tabulating them into one, the fre- 
quencies in the subtables would in effect be 
combined cell by cell in the compound table. 
Assuming constant working origins and equal 
intervals, any one cell in the compoune table 
will have the same deviation value as the cor- 
responding cell in each of the separate tables. 
That is, the functional value of a single case 
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in corresponding cells (in different tables) is 
the same cell by cell; and since all of the 
functions entering into the summations of a 
table are first degree with respect to frequency 
(the deviation values alone going to the sec- 
ond degree), the results for each cell or array 
are additive with respect to the frequency. In 
other words, a frequency of seven in a par- 
ticular cell in the compound table will con- 
tribute the same amount to Xx'y’ that separate 
frequencies of three and two and two, in the 
corresponding cell in three separate tables, 
will contribute when the =x’y’ values for the 
three separate tables are calculated and then 
added together; and the same is true of this 
frequency of seven with respect to the other 
five summations—x’*, Sx’, etc. 

Other ways of thinking about the combin- 
ing may be simpler. In combining two tables 
one may tabulate them together on a distri- 
bution sheet, using different colored pencils 
for the two different populations. If now one 
calculates summations for the tallies of one 
of these colors, and then for the tallies of the 
second color, and adds these summations to- 
gether, he will have the same totals as though 
he had counted both colors of tallies together. 
Yet the summations for each of the colors 
separately represent what he would have if he 
took summations from separate tabulations 
on separate distribution sheets. On the other 
hand, counting both colors together is what 
one naturally does when working from the 
compound table; so that the two processes 
are equal. Another way of seeing why sum- 
mations of the functions come out the same 
by either process is to imagine that each of 
the tables to be combined is on a sheet of 
glass. The several subtables can therefore be 
laid one over the other: the frequency in any 
cell of the compound table would be the 
total which one would get by looking down 
through the several layers of glass and count- 
ing the tallies one can see in that particular 
cell in the several superposed tables. In similar 
fashion, the various marginal summations for 
a compound table would be seen to be sums 
of the corresponding figures on the several 
contributing tables. The calculation of r from 
the combinations of the several corresponding 
major summations is then exactly what one 
would do if all of the layers of glass were 
coalesced into a single table. 

The relationship can be proved algebrai- 
cally, though this would seem to add little. 
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The proof amounts simply to expressing {. 
the frequency in the compound table, analyti. 
cally as the sum of the corresponding frequen. 
cies of the contributing tables. Thus, for any 
one array (i.e., the same array, with re spect 
to the working. origin, in all of the tables). 


fx’ = (f, + fp. . + fk)’ 


where the subscripts indicate the various ta- 
bles. Therefore, for the entire correlation 
table, 

fox’ = S(f, +f, +... +f,)x’ 


and the same thing is true for the other fiye 
summations ordinarily used. The additive 
character of these summation functions® with 
respect to frequency is thus seen. 

Thus far the discussion has dealt only with 
combining. Subtracting a population out is 
simply the reverse. That is, if one has a cor- 
relation table which consists of more than one 
identifiable group, he can take out any group 
and calculate the correlation for the remain- 
ing group by subtracting the summations for 
the subgroup from the summations for the 
entire group: the remaining summations lead 
directly to the correlation for the remaining 
groups. The group which is thus taken out 
may of course itself be a compound of several 
subgroups. Successive groups may be taken 
out as desired; each group which is taken out 
as a unit however requires separate tabulation 
and calculation of its six moment summations. 
The economy of the present method rests in 
not having to retabulate one of the groups 
each time; that is, in taking groups out, one 
will seek to take out (and tabulate separately) 
the smaller group each time, obtaining the 
summations for the largest possible group by 
subtraction instead of by retabulating. Once 
the worker senses the freedom which is ac- 
corded him by virtue of the additive character 
of the moment summations he is dealing with, 
he can readily work out the best means of 
manipulating the groups. 


ILLUSTRATION WITH DATA ON COLLEGE 
ADMISSION 


The data in Table I show how a number of 
combinations can be made economically if one 
plans his work with this in mind. The corre- 
lations represent the relationship between 


® These six major summations for a correlation table are, 
strictly, moments, N being regarded as the zero moment (i.¢., 
4, given an exponent of 0 and becomes 1) and Exy being 
he product moment. It may be somewhat more ‘omen 
Sooter to refer to them as the 
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CORRELATION COEFFICIENTS 


TABLE I 


CORRELATIONS BETWEEN Four-YEAR HIGH SCHOOL AVERAGES AND AVERAGE GRADES 
IN FRESHMAN WORK AT THE UNIVERSITY OF CHICAGO 


oeation of High School r N 

“North Central States 
OS eae . 5622 553 
Indiana_ - . 6210 261 
Ohio . 5019 83 
ee es . 6565 98 
Wisconsin. _....-.--- . 5197 111 
Minnesota . 5529 23 
Iowa... . 6122 194 


=x’y’ =x" Sy" Zz Ly’ 


5,689 10,580 
3,070 4,545 
774 1,457 
1,052 1,554 
790 1,576 


9,409 : 289 
5,536 336 
1,695 . 93 
1,670 2 —10 
1,467 — § 


313 680 400 —_ —50 


2,139 3,607 





g All N. C. States* . 5750 1,323 
9. Atlantic States_....------ 8366 110 
10. Southern States _4756 106 
11. West Central States._.... 4996 168 
12. Mountain and Pacific 
. 6156 93 


3,728 190 


13,827 23,999 23,905 
724 2,497 1,875 — —57 
907 1,614 2,200 : 133 
1,470 3,689 2,591 - 35 


917 ; , —13 





All States* . 5463 
Chicago City . 6120 


1,800 
1,707 


33,378 
30,081 


17,845 32,017 260 941 
17/063 





Total Admissions- - - - . 5618 


*Exclusive of high schools in Chicago. 


3,507 


four-year high school averages and averages 
made in the freshman year at the University 
§ Chicago, for all high school students who 
entered the University during a five-year 
period. The six summations shown for each 
row are values that one needs for the calcula- 
tion of each r; and the adding of these values 
(vertically) gives the required summations 
for the correlation of a combined population. 
Separate correlation tables were initially pre- 
pared for each state. The correlation for the 
North Central states (line 8), for each of 
the groups of states (lines 9-12), for all 
states together (line 13), and for all admis- 
sions (line 15), were obtained without addi- 
tional tabulation. Means and standard devia- 
tions for these groups were obtained in the 
same fashion. Thus, working with a total 
population of 3,500 cases (which had to be 
tabulated once, in various tables), the com- 
binations shown were obtained with a saving 
of 7,100 case-tabulations over what would 
have been necessary if a new table had been 
made for each combination. 

As an illustration of separating out popula- 
tions, assume that one wished the correlation 


*These data are from the writer’s thesis previously cited. 
They have not heretofore been published, with the exception 
of the correlation for Chicago high schools which a 
in the following reference: Douglas E. Scates. ‘A Study of 
High-School and First-Year University Grades.” School Re- 
view 32:182-92; March 1924. The data for out-of-Chicago 
bigh schools are for admissions during the five years 1919, 
1920, 1921, 1922, and 1923; the data for the Chicago high 
schools are for admissions during the five years 1916, 1917, 
1919, 1920, and 1921. 


28,979 1,719 —441 


34,908 63,459 60,996 1,979 500 


for all states together except Illinois. This 
value could be obtained by subtracting the 
summations for Illinois (line 1) from the 
summations for All States (line 13) and cal- 
culating the correlation from the remaining 
values—a few minutes’ work, in lieu of tabu- 
lating 1,247 cases. Or, if one had not formed 
an All States total, he would add the summa- 
tions for Illinois and for the City of Chicago 
(line 14) and subtract the total of these from 
the summations for Total Admissions (line 
15). 


APPLICATIONS AND PRACTICAL 
CONSIDERATIONS 


There are a great many ways in which one 
may subdivide and regroup correlation popu- 
lations. The procedure described herein is 
adapted to combining or separating tables on 
the basis of a third variable (usually cate- 
gorical), when the population is distributed 
on the same two variables in all of the tables. 
The division of the total group into tabula- 
tions according to categories or intervals of a 
third variable is equivalent, in principle, to 
partial correlation: but combining tables is 
not equivalent to multiple correlation. The 
latter assumes that various values are com- 
bined for each individual case before that case 
is entered into a single table; in the present 
paper we are discussing the compounding of 
entire tables, not individual cases. Subdivid- 





— 
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ing a population on the basis of values in one 
or both of the two traits on which it is dis- 
tributed, leading to “subcorrelations,’”® is 
analogous to the present procedure only if one 
assumes that division on a particular third 
variable leads to contiguous but  non- 
overlapping distributions on one or both of 
the two traits of the original correlation table. 
The procedure herein described has no rela- 
tion to correlations between one or more com- 
ponent elements of a trait and the composite 
of all the elements—which normally utilizes a 
single population, and does not involve sub- 
groups.° 

The method will be of some help in suc- 
cessive regrouping of the data to accomplish 
either simple or joint subdivisions. That is, 
one may wish to break down a population on 
the basis of such traits as sex, age, location, 
etc. If these divisions are to be made inde- 
pendently, one will start with the original 
table for each breakdown; if they are to ac- 
complish a joint subdivision, one will continue 
from the last preceding separation. In either 
case the procedure will save at least half of 
the retabulating for any dichotomous trait, 
and will save some work in other cases. If one 
foresees such analyses, he will probably pro- 
ceed in the other direction, making separate 
tabulations for groups which are homogeneous 
with respect to sex, age, location, etc.; their 
successive combination into any number of 
compound tabulations is then simple and ex- 
peditious, involving no further tabulation. If 
one wants each correlation table fully tabu- 
lated, so that he can see the complete two-way 
distribution, then there is no saving to be 
found in the methods being discussed; but 
they will afford essential checks on one’s 
tabulating and calculations, and may be em- 
ployed solely for that purpose. 


5 Dwyer, P. S. ‘‘The Use of Subcorrelations in Determining 
the Predictive Power of High-School Grades,’’ Journal of Edu- 
cational Psychology, XXVIII (December, 1937), pp. 673-80. 

* Harris, J. Arthur. ‘“‘The Correlation between a Component, 
and between the Sum of Two or More Components, and the 
Sum of the Remaining Components of a Variable,”’ Journal of 
the American Statistical Association, XV (December, 1917), 
pp. 854-59. 

Ghiselli, Edwin E., and Kuznets, George, “‘Short-Cut Methods 
for Calculating Raw and Corrected Correlations Between a 
Composite Variable and Its Components,’ Journal of Educa- 
tional Psychology, XXVIII (March, 1937), pp. 237-40. 


Conrad, Herbert S. ‘‘On the Calculation of the Correlation 
Between a Single Element of a Composite and the Remainder 
of the Composite,’ Journal of Educational Psychology, XXVI 
(November, 1935), pp. 611-15. 


Edgerton, Harold A. ‘‘A Formula for Finding the Average 
Correlation of Any One Variable with the (n—1) Other 
Variables Without Solving Any of the Individual Correlations,” 
of Educational Psychology, XXVI (May, 1935), pp. 
373-76. 


‘final calculation. Both procedures yield the 
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For those who are accustomed to Utilize 
punch cards and Hollerith tabulating ma. 
chines for the calculation of coefficients of 
correlation it may be pointed out that the 
combinations set forth herein may be obtaine; 
more quickly than cards can be sorted and 
tabulated on the machines to effect combina- 
tions of populations. Likewise when a large 
population has been correlated and is to be 
broken down into two very uneven groups 
it may be found more expeditious to sort oy 
the smaller group and obtain the correlation 
summations from that group on the machines 
getting the summations for the larger sub- 
groups by subtracting the small group sum- 
mations from those of the total group. This 
is true even though one may be interested 
only in the larger of the subgroups. 


Another situation exists when one has the 
means and standard deviations fully calcu- 
lated, for each constituent population. For the 
present method this is no advantage, as com- 
pleted constants are not utilized. The con- 
stants could be converted into summations. 
by employing N, but they would be summa- 
tions around various true means and not 
around a common origin. They would there- 
fore call for a correction (back to a common 
origin), or, if combined directly to form com- 
pound xy, =x’, Sy, and N, the final formula 
would involve two differences for each table 
(between its means and the means of the 
compound table). Starting with fully calcu- 
lated constants is the situation for which Dun- 
lap’s formulas were derived, and if one begins 
with data in such form, he will be doing 
practically the same thing no matter which 
procedure he follows. 


The theoretical’ or structural difference be- 
tween Dunlap’s method and the present one 
is that his procedure combines summations 
after they have been corrected to the subgroup 
means and then to thé compound group 
means, whereas the present method combines 
them when taken around a common working 
origin, and corrects the summations to the 
compound group means in one step, in the 


same results;’ the significant difference is 
purely a practical one—in what form are the 
data to start with? If one has the necessary 


™While Dunlap did not extend his formulas to cover sub- 
traction of groups, this result is accomplished by subtracting 
(instead of adding) a set of parameters for the population in 
question, in both the numerator and the denominator of the 
formulas he presents. 
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<ymmations, the present method obviates cor- 
recting them to the subgroup means and it 
corrects them to the compound group means 
in a single calculation instead of separately 
for each subpopulation. It does. not entail the 
calculation of means or standard deviations 
for either the subgroups or for the compound 
group if these are not desired in themselves, 
for the method does not make use of them. 
If one has not planned his work so that the 
working origins and intervals are equal from 
table to table, and adjustments for these in- 
equalities are necessary, the operations be- 
come more involved, and Dunlap’s formulas 
may call for less calculation. It should be 
considered, however, that the present proce- 
dure permits readily of further combining and 
separating beyond the first operations, and 
after adjustments have once been made to 
bring working origins or class intervals in line 
these are no longer an obstacle. The formulas 
involving Nroo, or Sxy, on the other hand, 
require the calculation of new means for the 
compound group, and the consequent changing 
of two differences in means for each table, for 
every subsequent combination or separation 
of tables. 


ADJUSTING FOR DIFFERENT WORKING 
ORIGINS 


If the same working origins were not used 
in calculating the deviations on the various 
tables, the summations for each population 
can be adjusted by the following procedure. 
If the origins actually used for a particular 
table are to be moved by the amount d, or d, 
in order to make them the same as the origins 
for other tables, the summations for the table 
will have the following corrections added to 
them (the sign of d varying inversely with 


the direction of the shift). 

To Sx’y’ add:* d,&x’ + d,3y’ + Nd.d, 

*If one is using a correlation formula employing a differ- 
ence term in the numerator, as >(x’ —y’)?, he may make 
the adjustment for a change in origin in any of three ways 
It seems most expeditious to adjust for several terms of the 
numerator jointly. Since Dx’y’ = (%4) (Lx? + Ly? — T(x’ — 
y’)*], assuming the same origins and intervals, the adjustment 
given for Sx’y’ may be applied directly to the three terms 
in the bracket as a unit, instead of adjusting them individually. 
It is permissible to do this for each separate table, before the 
values are combined with those of other tables, if one con- 
tinues all the way through to treat this combination as a unit. 
Note that if this part of the numerator is not divided by two. 
twice the adjustment stated will be called for. The other term 
of the numerator, usually }x’Sy’/2N, receives its attention 
in the final calculations, after Sx’ and Sy’ for the individual 
tables have been adjusted and combined. 

A second form of adjustment for the term S(x’ —y’)? is 
to make an arithmetic combination of the values worked out 
for the first three expressions given in the list: since 
L(x’ — yy’)? = Sx’? — 2Ex’y’+ Sy”, the desired adjustment 
may be obtained by adding together the adjustments calcu- 
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To 3x” add: 2d,3x’ + Nd,? 
To Sy” add: 2d,Xy’ + Nd,’ 
To 3x’ add:** Nd, 
To Sy = add:** Nd, 


The sign of d must be preserved: if the new 
origin is smaller than the old, d is positive; if 
the shift is to a higher point on the scale, the 
amount of the change will have a negative 
value in the above expressions; but note that 
Ndd and Nd? will always be positive. The 
value of d must be expressed in the same 
units that the summations are: if they are in 
class intervals as units, then d must be simi- 
larly expressed before substituting in the cor- 
rections. The values and signs of d, and d, 
will of course be independently determined, 
according to the amount and direction of 
shift. If only one origin needs moving, to 
make it the same as the origins used in other 
tables, then the d for the other origin in that 
correlation table is zero, and two of the above 
five adjustments become zero, and the one for 
the product summation is reduced to a single 
term. The =x’ and Sy’ which are called for in 
the first three adjustments are used just as 
they are obtained from the correlation table— 
that is, the values for =x’ and Xy’ are not 
first corrected according to the two lower cor- 
rections, as that has been done already in the 
algebra. =x’ and Sy’ adjusted by adding Nd 
to them are employed only for combining di- 
rectly with other similar summations from 
other tables. The adjustments shown above 


lated for =x’? and Sy’? and subtracting from them two times 
the adjustment for =x’y’. A third method is to make this 
combination algebraically and calculate its value directly: 
2(Sx’ — Sy’) 4, —4,) + N(d, — d,)?. If one is using a 
formula for r which contains a sum term, as 3(x’ + y’)?, 
similar plans may be employed, with appropriate changes in 
signs. 

The symbolism Sd is sometimes written in lieu of 
=(x’ — y’)*. Such a d has no relation to the d used in this 
paper to represent the amount of voluntary shift in origins. 


** The worker may note that, if d is taken equal to the 
distance from the working origin to the mean (M A), the 
value of the adjustment for =x’y’ becomes equal to the cor- 
rection usually made in the product term (namely, }x’Sy’/N) 
to correct it to the means of the group. The same thing is 
true of the adjustment made in the square terms of the 
denominator. Such equivalence shows the relationship between 
these formulas and the usual corrections, the present ones 
being more general; the two are equal only when d has the 
value stated, whence d. = Yx’/N and d, = Sy’/N. One 
may note further that, when adjusted, the full value of Dx’y’ 
— (=x’Sy’)/N remains the same as before the adjustment— 
and that the value of the denominator remains the same. 
This is as it should be—the correction must vary properly 
to offset the effect of changes in origin on the summations. 
The principal terms and the =x’ and Sy’ must all be ad- 
justed for each table however, in order for them to have the 
proper weight in the compound. 

The value, Sd, as called for by the Conten-Dantte hand 
and machine correlation charts, and possibly others, has no 
relation to the d used in the present paper, but stands for 
Z(x’—y’), or Dx’ — Ly’. When it is called for, adjust- 
ments to it will automatically be obtained by adjustments to 


, 


Sx’ and Sy’. 
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are essentially Charlier’s expressions for 
checking by changing origins. 

The summations may of course be adjusted 
for transferring their origin to any desired 
point, just so all the X variables in the several 
tables to be combined or separated have the 
same origin, and all the Y variables have the 
same origin. One may, if he desires, calculate 
the means for the compound table and use 
these as working origins. Little however is 
gained by so doing. 


ADJUSTING FOR DIFFERENT CLASS INTERVALS 


If the several tables which are to be com- 
bined or separated have class intervals of 
different widths for the same variable, adjust- 
ments can be made in the moment summa- 
tions by applying a ratio. The value of =x’ 
based on a class interval of 1o will (theoreti- 
cally, at least) be just half as large as =x’ 
based on an interval of 5, and the value of 
=x”? will be one-fourth as large. To make 
changes in the summations for changes in the 
class intervals, one may proceed as follows: 

If h, and hy represent the class intervals 
used in any table, and i, and i, represent the 
intervals which it is desired to change to, 
derive the ratios h,/i, and h,/iy. For sim- 
plicity of symbolism let these ratios be repre- 
sented respectively by a and b. Then 

Multiply %x’y’* by ab 
Multiply =x’* by a? 
Multiply Sy? by b? 
Multiply =x’ bya 
Multiply Sy’ by b 


In other words, multiply the observed sum- 
mation by the ratio of the width of its own 
class interval to the width of the desired in- 
terval, using the same degree of the ratio as 
the deviations have. 

If one has to adjust the summations from a 
table for both class intervals and origins, it 
seems more straightforward to adjust for ori- 
gins first, expressing d in terms of the old 
intervals. If however one makes the above 
adjustment for intervals first, the resulting 
=x’ and Sy’ values will then be used in mak- 
ing the adjustments for changes in the origins 
as given earlier; also d will be expressed in 
terms of new intervals. 


* The adjustment of £(x’—y’)? may be accomplished by 
multiplying it by ab and then adding to it the following: 
atx + b*Ly’® — ab(Sx? + Ly”). As in the case of adjust- 
ing for a change in origins, however, it is simpler to treat 
that part of the numerator, (%)(Zx’? + Dy’*— E(x’ — y’)?], 
as a unit; the adjustment then becomes the same as for 
=x’y’, namely, multiplication by ab. 
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ELIMINATING HETEROGENEITY: C ONTROLLIN¢ 


-orrect 
VARIATION IN MEANS OF SuBcRoUps eon 


correc! 


The procedure described in this Paper js Differe 
well adapted to accomplishing certain special for as 
purposes. Differences in means of subgroups 3x’ te 
may contribute positively or negatively (a¢. rectior 
cording to their position) to the correlation rectiOl 
for the compound population—so that they than 
may cause the resulting r to be higher or there 
lower than any of the initial r’s. If the com. true | 
bined population is regarded as homogeneoys obser’ 
for one’s purpose, this effect on the correla. Th 
tion is accepted as a part of the normal situa. rectin 


tion. If the combined population is not rectin 
homogeneous for one’s purpose, one may wish group 
to remove from the compound correlation the ing, 
effect of differences in means of the subgroups. ment 
This can readily be done while carrying (The 
through the procedures outlined. It can be be ca 
done in one direction only, or for both of the resull 
traits at the same time. tain 


To remove the effect of differences in means X mi 
in one direction only (say the x means) of a is mé 
number of subpopulations which are to be table 
combined, one corrects the moment summa- gin i 
tions containing x for each individual table, meat 
back to the mean of that table as origin, there 
before combining the summations of the sev- num 
eral tables. For example, using the traditional case 
correlation formula, one would apply to each latio 
individual table the full correction® for the nu- arbi 
merator, =x’Sy’/N, and also the correction for com 
=x” in the denominator, namely, (%x’)?/N. equé 
The summations for the several tables to be desi 


combined would then consist of: Sxy, 3x’, bles 
xy’, Sy’, and N. (Note the first two terms will 
do not carry primes and are deviations from 2x’, 
subgroup means.) After the adding together The 
of these values for the several tables, r is rect 
calculated from the formula Just 








7 inte 
r=——__—— A : inte 
Vix? Vy’? — (Sy’)?/N obt: 


Differences in working origins for the X vari- e 


able, from one table to another, need not be din 


_ © While this full correction does have the effect of correct- het: 
ing the numerator for differences in means in both directions, ee 
it is nevertheless not an excessive correction for one direction 
only. This follows because the correction for one direction 
only is the same as the correction for both directions. That 
is, when deviations are taken around the true mean of one 
trait, the entire correction becomes zero, regardless of whether 
the origin for the second trait is taken at the mean or else- 
where. This is obvious from the formula: if either =x’ or Dy’ 
becomes 0, the entire correction, }x’Sy’/N, becomes 0. This 
condition may be rationalized by noting that shifting the 
population orthogonally—along the direction of either axis 
(as when one origin is not at the mean)—does not have any 
effect upon the correlation, and hence does not affect the 
numerator. 
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corrected for, since the x’ suminations are 
corrected to the mean of each table as origin. 
Differences in class interval will be adjusted 
for as previously described, except that the 
sx’ term has become zero and needs no cor- 
rection. It may be noted in passing that cor- 
rections could be made to some other point 
than the subgroup mean in the event that 
there was reason for doing so—such as when 
true subgroup means were known and the 
observed means were desired equated to them. 

The above formula has the effect of cor- 
recting Sx’? to the subgroup means and cor- 
recting Sy’ to the mean of the compound 
group—the former being done before combin- 
ing, and the latter after combining the mo- 
ment summations from the several tables. 
(The signs of the various summations must 
be carefully preserved in the combining.) The 
results will be the correlation one would ob- 
tain if the several tables showed identical 
X means, but differed in their Y means. This 
is not the same thing as though all of the 
tables to be combined had their working ori- 
gin in the X direction shifted to the actual X 
mean of the compound table—it is true that 
there would be no corrections called for in the 
numerator or in the =x*, as in the present 
case; but shifting a working origin has no re- 
lation to shifting actual means. Shifting the 
arbitrary origin has no effect upon the out- 
come; shifting the means so that they are 
equal produces a changed situation. If it is 
desired to equate both means of all the ta- 
bles, then all three of the usual corrections 
will be made for each table and only Sxy, 
Xx’, Sy’, and N will remain to be summed. 
They will yield r directly without further cor- 
rections of any kind for origin, though ad- 
justments for differences in size of class 
interval may be called for if different class 
intervals were used from table to table in 
obtaining the original subgroup summations. 

One will make assumptions of these kinds, 
and resort to these procedures, when he de- 
sires to avoid introducing inappropriate 
heterogeneity into his compound popuiation. 
It is a matter of somewhat difficult judgment 
to determine when this situation arises. One 
is always faced with the question, If the 
groups are not homogeneous when combined, 
why combine them at all? Why not calculate 
the separate correlations and stop? The com- 
bined population may iron out erratic fluc- 
tuations in a smaller group; but it may also 
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cover up distinctive tendencies of distribution 
in the separate groups. The utilization of 
combining formulas therefore for groups 
which do not naturally belong together—es- 
pecially when corrections are made for dif- 
ferences in one or both means—must be done 
after careful consideration. For example, one 
may have separate correlation tables for boys 
and girls. Boys and girls form a natural popu- 
lation when combined, and combining the 
tables is therefore legitimate. But to equate 
the means of the two groups before combin- 
ing, by the methods just described, would 
raise a serious question. The result is a cor- 
relation that would obtain if the two sexes 
were equal on both means: but they are pre- 
sumably not, and one therefore has the obli- 
gation of explaining why that correlation is as 
acceptable as the compound correlation based 
on observed (unequal) means. 

The method of equating one or both means 
is available for partialling out the influence 
of an additional variable when that variable 
is chiefly responsible for changes in the means 
of one or both of the traits entering into the 
correlations. Thus, time may be held con- 
stant, when it is known to be the chief factor 
influencing the means of successive groups, 
and when the concommitance of variations in 
short intervals of time is of chief concern. 
Similarly age, or grade levels, of different 
groups of pupils may be held constant—if 
there is justification for so doing in a com- 
pound table. This type of partial correlation 
is not restricted to a quantitatively expressed 
third variable, nor to a linear distribution of 
means of subgroups. The individual distribu- 
tions for specific values of the third variable 
give analytic, specific correlations; the com- 
pound correlation gives an average, similar to 
any partial correlation. It may be used where 
averages are what is wanted. 


CONTROLLING VARIATION IN DISPERSION 
WITHIN SUBGROUPS 


The standard deviations of groups enter- 
ing into a compound correlation do not raise 
or lower the resulting correlation directly, as 
means may do, but have the effect of weights. 
That is, they behave much like N does in 
this regard. They could not produce a com- 
pound correlation greater or smaller than any 
of the original correlations, as differences in 
means can; they simply weight more or less 





—————————————— 


t 


i 
: 
7 
; 
j 
’ 


282 JOURNAL OF EXPERIMENTAL EDUCATION 


the degree of correlation which is contributed 
by any one subpopulation. Being able to con- 
trol the standard deviations may be of larger 
interest through enabling one to equate dif- 
ferent scales of measurement that may have 
been used. 

Equating, or removing the effect of varia- 
tions in, the standard deviations of the dif- 
ferent subgroups may be accomplished during 
the process of combining, and is similar to 
changing the value of intervals. The standard 
deviation in one trait only, or the standard 
deviations of both traits, may be controlled. 
First, we may assume that one wishes also to 
hold the subgroup means constant, and that 
the summations for each subgroup have been 
corrected to the means of that group. Then, 
to equate the standard deviations for one 
trait only, Sxy and x? for each subgroup 
may be divided respectively by ox and o,” 
for that group.® To control both standard 
deviations, Sxy is divided by the product of 
the 2 o’s, and =x? and Sy* are divided re- 
spectively by o,” and o,*—giving V N*, or N, 
in the denominator. These divisions are made 
for each subgroup. Inasmuch as the subgroup 
standard deviations have all been made unity 
by these divisions, only two values need to 
be combined in order to obtain the compound 
correlation: N and xy. The correlation co- 
efficient for the compound group is given by: 
r= Sxy/N. 

It is of course possible to equalize standard 
deviations by reducing all of them to some 
other value than unity; in this case the de- 
nominators will not reduce to N. It is also 
possible that one would not wish to equalize 
them, but to modify only selected ones. This 
offers no technical difficulties. All, or selected 
groups, may be multiplied by any desired ra- 
tio—so long as the ratio is squared for multi- 
plying terms in the denominator which are 
under the radical. 

In the foregoing the summations were all 
corrected to the subgroup means, which has 
the effect of equating the means. One may 
however wish to modify intragroup dispersion 
alone, without changing the means. Practi- 
cally, the best procedure is to go through the 
foregoing steps for each group and then shift 

* Subdividing Ex? by o,? yields N, the population of the 


subgroup. In reality therefore the division need not be ac- 
complished except for purposes of checking; N may simply 
be substituted for Sx?. For all of the subgroups combined, 
this gives the total population, which is used in place of >x* 
in the final calculation of r. The formula becomes: 
r= Sxy/VNZy"*, or xy/No,. 


[ Vol. gd, No ? 


the summations concerned back to the ev 
mon origin for all groups, by adding le 
each subgroup) =x’Sy'/N back to the mod; 
fied Sxy, in order to create a modified Sey’ 
for the subgroup, and by adding ( Sx')2 
back to the modified 3x*. All six of the sum 
mations for the various groups are then com 
bined and r may be calculated. The ressen 
for shifting the summations from the common 
origin to subgroup means, and then moving 
them back again to the common origin after 
multiplying or dividing, is that the deviations 
from an arbitrary origin contain a portion 
which must not be altered. While the sy" 
and =x’ may both be changed by the same 
ratio for a single distribution, without error 
they lose their proper weight when they are 
combined into a compound of summations. 

There is one caution which should be ob. 
served. When one multiplies the moment sum- 
mations for a given correlation he is doing 
more than changing its weight among a group 
of correlations. He is also multiplying the 
random errors in the original data and prob- 
ably will not obtain the same numerator that 
he would from actual observations which had 
the same standard deviation as that to which 
he haz mechanically changed his summations. 
This error may, under some conditions. be 
offset by utilizing Kelley’s formula for the 
effect of range of talent on r, or Pearson's 
formula for curtailed distributions.’ From 
one of these formulas, the appropriate sub- 
group r for the changed range may be ob- 
tained from the actual r for that table; when 
this is multiplied by Noo (using the modified 
o’s which may have taken the value of 1), 
the desired modified Sxy is obtained for en- 
tering into the compound. If one is reducing 
standard deviations, the xy will be smaller 
than the one which would be obtained by 
—" reducing the original Sxy in propor- 
ion. 


ALTERNATIVE PROCEDURES 


For the purpose of simple combinations, the 
tabulating of separate tables for all desired 
combinations, either by hand or machine, and 
the use of Dunlap’s combinative formulas, 
have already been commented on. It remains 
to note alternative procedures for holding 
means or standard deviations constant. Again, 
Dunlap’s formulas are available; to hold the 


Kelly, Truman Lee. Statistical Method (New York: M 
millan Co., 1923), pp. 221-30. 
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subgroup means constant, simply eliminate all 
of the terms containing delta in his formula 
:2: that is, the delta’s all become equal to 
zero. The delta’s may also be changed to 
some other value if desired. To equate or 
otherwise modify standard deviations, multi- 
oly the appropriate terms in the numerator by 
the desired ratio, and the appropriate terms 
in the denominator by the square of the ra- 
tio. Other special results as described in the 
present paper can be secured from his for- 
mula. The relationship of his formula to the 
procedure in the present paper should be clear 
when it is recognized that multiplying out 
such a term aS Moxy, oy,,.Tx,.ym Zives =xy for a 
subgroup; adding mé,,A4,, shifts the origin to 
the compound group means. Similarly in the 
denominator, mo,,,* is x? for a subgroup, and 
adding mé8,,? shifts the origin to the compound 
group mean. While the two procedures are 
interchangeable at various points, they start 
with different material, and make different de- 
mands. Dunlap’s formula assumes completed 
calculations for all of the tables; the method 
of the present paper is based on raw summa- 
tions, and in some cases tabulations may be 
entirely avoided. 

May" presented a method of obtaining r 
when one starts with a compound table (or 
the essential parameters from it) and wishes 
to hold the means of the subgroups constant, 
without further tabulating. His method does 
not require linearity of the means, but re- 
quires their calculation. The method supple- 
ments that of the present paper, for it is 
adapted to use with (the results of) a com- 
pound correlation table and does not require 
any of the constituent tables, whereas the 
method herein described employs summations 
from the individual tables entering into a 
compound, and does not require a compound 
table. The choice between methods will there- 
fore depend on where one is beginning in his 
work. May’s method does not allow for modi- 
fying the individual standard deviations, and 
does not necessarily produce any tabulation. 


McCloy”? presented a method of obtaining 
a compound of analytical correlations which 
equates both the means and the standard de- 


"May, Mark A. “A Method for Correcting Coefficients of 
Correlation for Heterogeneity in the Data,’ Journal of Edu- 
itional Psychology, XX (September, 1929, pp. 417-23. 

*McCloy, C. H. “A Method of Computing Partial Corre- 
lation and Regression Equations with Variables Having Cur- 
vilinear Intercorrelations,’ Journal of Educational Research, 
XVI (November, 1927), pp. 285-95. 
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viations of the subgroups. It does not assume 
linearity, and could be adapted to equate only 
the means or only the standard deviations, as 
desired. The procedure builds up a compound 
correlation table through special tabulating 
so that means and standard deviations are 
held constant by the method of tabulating. It 
does not yield separate correlations for the 
various subgroups unless these are especially 
made. It will however yield eta. 


Partial correlation, while usually thought of 
as analytical rather than combinative, is nev- 
ertheless an average—an average of specific or 
analytical correlations, with means equated; 
it is therefore comparable to the procedures 
described herein. It is definitely limited in 
two ways—through requiring linearity of 
means of subgroups when distributed on any 
pair of traits, and through requiring that the 
third variable be quantitative. It is true that 
arbitrary numbers may be used if the third 
variable is categorical, but these do not give 
satisfactory results unless they are so chosen 
as to lead to linear distributions of the means 
of subgroups or classes created by this third 
variable. Without this linearity the means of 
the classes are not held constant. The method 
does not lend itself readily to equating or 
otherwise modifying standard deviations. It 
is the only method of those discussed which 
is likely to prove practicable when mere than 
four variables are employed. On the other 
hand, when the third variable is quantitative 
and linearity is fulfilled, the method is 
adapted to expressing the average relationship 
between the third variable and the first, for 
various specific values of the second trait. 
The methods already described do not yield 
such comparisons. For traits one and two, 
however, additional variables can be held con- 
stant by one method as well as the other. 

All of these procedures will give the same 
results when the conditions which are as- 
sumed are fulfilled. 


SUMMARY 


The correlation coefficients, standard devia- 
tions, and means may be obtained for com- 
bined or separated populations by adding to- 
gether (or subtracting) the moment summa- 
tions, for the several constituent groups, which 
are ordinarily obtained for individual corre- 
lations: Sx’y’ or =(x’ — y’)?, Sx”, Sy’?, Sx’, 
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Sy’, and N. After such combination (or sub- The method is well adapted to accomplish. 
traction) of the summations the results are ing certain special purposes, such as eliminay. 
substituted in the usual formula for calculating ing variation in one or both means, and op, 
r from arbitrary working origins—or in the or both standard deviations, of the severa| 
appropriate formulas for the standard devia- populations which are to be combined. They 
tions or means. The procedure is not restricted can be equated (held constant) by correct 
to any particular correlation formula, nor to appropriate moment summations for each 
distributions which have been tabulated in population before these are combined. This 
two-way tables. It is somewhat more facile, has the effect of giving an average correlat 
when the necessary data are available, than for the several groups as though one or both 
are combining formulas which utilize directly of the means or standard deviations wer, 
r, , and M from each distribution. equal. These are forms of partial correlatio: 

If the working origins and class intervals which are not dependent upon a quantitatiy 
are not the same for the tabulations of the — third variable, nor upon assumptions of linear- 
several constituent populations, adjustments ity of the distribution of means of 
may be made for these differences. subgroups. 


Lut 





